Before I answer this key question, it is important to define and differentiate between "responsibility" and "accountability." These terms are so often used interchangeably, but understanding the difference can help a project manager to succeed when managing a project team.
In Traditional / Plan-driven project management, responsibility is when a team member is assigned an activity to be accomplished (having his/her hands-on) within a defined timeline. On the other hand, if this same team member will be questioned about the status of the activity assigned, then the team member is also held accountable.
In an Agile / Adaptive environment, project will only succeed if the whole team is accountable rather than only responsible for doing tasks. Accountability represents quality in an agile environment. Now, back to the main topic, as a project manager, what is the best way to build team accountability? The best way is by involving the team in decision making, by encouraging the team members to self-organize in determining what work needs to be done, in determining how to perform the work and who should do what.
As a project manager or scrum master your responsibility is to make sure that the team is working in a healthy environment and to promote courage, commitment, respect between the team and most importantly openness. Let the team to focus on achieving the sprint goal and materializing the project vision. Doing that the team will feel fully accountable for the project and they will feel ownership for the decisions and the project as a whole .