Increasingly, teachers have access to technology applications that support and extend learning. These applications can be ubiquitous, instructional or STEM-based. Existing classroom observation protocols tabulate technology use but do not capture the quality of use in the context of research-based instructional best practices. Building on our published evaluative framework, we are developing an observation protocol that evaluates the quality of teachers’ integration of technology applications with (1) science and engineering practices as described in the Next Generation Science Standards, (2) student-centered teaching, and (3) contextualized teaching practices. The protocol considers diverse types of technology applications and ways to define quality of technology implementation. This paper will share our approach to iteratively developing the protocol to define concrete indicators of the quality of technology use in classrooms—including piloting in varied high school classrooms to ensure validity and applicability—and will highlight challenges and specific examples. Eval172017 Conference