The qualitative study aims to address the current understanding of English language assessment of both the teachers and students in the secondary schools in Bangladesh. The study conducted semi-structured interviews with six English teachers and focus group interviews with two groups of students in two different secondary schools and these interview responses were compared to probe the related understandings, challenges and experiences of both the teachers and the students. Findings reveal that the present English assessment system in the secondary level in Bangladesh does not reflect a balanced development of all the language skills of the learners and there are inconsistencies between the stated objectives of teaching English and the actual teaching methods in the language teaching in the secondary schools in Bangladesh. Though summative assessment is still dominating, the practice of formative assessment is slowly developing. It also indicates that there has been a gradual shift in the assessment process and the teachers were trying to use individual assessment strategies to motivate students’ learning. Both the teachers and the students in the study emphasized that current assessment is mainly based on reading and writing. However, for overall development of language skills, the secondary English language curriculum may need to be redesigned to include all the four skills in the assessment system. Better opportunities for training to develop teachers’ effectiveness and their knowledge of learners should be also considered.