In this study, effectiveness of a computer science course at the secondary school level is investigated through a holistic approach addressing the dimensions of instructional content design, development, implementation and evaluation framed according to ADDIE instructional design model where evaluation part constituted the research process for the current study. The process has initiated when the computer science curriculum had major revisions in order to provide in-service teachers with necessary support and guidance. The study is carried through as a project, which lasted more than one year and both quantitative and qualitative measures were used through a sequential explanatory method approach. The intention was to investigate the whole process in detail in order to reveal the effectiveness of the process and the products. In this regard, not only teachers' perceptions but also students' developments in their perceptions of academic achievement and computational thinking, as well as correlations between the computational thinking sub-factors were investigated. The findings showed that the instructional materials and activities developed within the scope of the study, positively affected the computational thinking and academic achievement of students aged 10 and 12 years old. The teachers' weekly feedbacks regarding application structures and implementation processes were also supported the findings and revealed some more details that will be useful both for instructional designers and teachers.