The next generation of medical information system will integrate multimedia data to assist physicians in clinical decision-making, diagnoses, teaching, and research. This paper describes MIARS (Medical Image Annotation and Retrieval System). MIARS not only provides automatic annotation, but also supports text based as well as image based retrieval strategies, which play important roles in medical training, research, and diagnostics. The system utilizes three trained classifiers, which are trained using training images. The goal of these classifiers is to provide multi-level automatic annotation. Another main purpose of the MIARS system is to study image semantic retrieval strategy by which images can be retrieved according to different levels of annotation.