Abstract

The American Society for Microbiology's curricular guidelines for Introductory Microbiology highlighted key laboratory skills in the isolation, visualization and identification of microorganisms as core learning objectives in the discipline. Since the publication of these guidelines in 2012, there has been a paucity of diagnostic assessment tools in the literature that can be used to assess competencies in the microbiology laboratory. This project aimed to establish a laboratory competency examination for introductory microbiology, with tasks specifically aligned to laboratory skills and learning outcomes outlined in curricular guidelines for microbiology. A Laboratory Competency Examination assessing student skills in light microscopy, Gram-staining, pure culture, aseptic technique, serial dilution, dilution calculations and pipetting was developed at The University of Queensland, Australia. The Laboratory Competency Examination was field-tested in a large introductory microbiology subject (∼400 students), and student performance and learning gains data were collected from 2016 to 2017 to evaluate the validity of the assessment. The resulting laboratory assessment is presented as an endpoint diagnostic tool for assessing laboratory competency that can be readily adapted towards different educational contexts.

INTRODUCTION

Background

The American Society for Microbiology (ASM) concept-based microbiology curriculum was published in 2012, which highlighted overarching concepts, fundamental statements and key competencies in scientific thinking and laboratory skills for an introductory microbiology curriculum (Merkel 2012). Recent studies in this space have focused on validating microbiology concept inventories that have been derived from these curricular guidelines (Paustian et al. 2017; Seitz et al. 2017), but to date comparatively little has been written on tools and instruments that can be used in developing key competencies in microbiology. Outside of knowledge retention relating to key concepts, the assessment of scientific thinking and laboratory skills largely relies on in-class quizzes, recordkeeping notebooks, and written reports (Rybarczyk, Walton and Grillo 2014; Shapiro et al. 2015). The majority of instruments used in assessing novel laboratory modules in microbiology revolve around survey instruments measuring student perceptions and learning gains (Shortlidge and Brownell 2016), rather than hands-on competencies in the laboratory.

Laboratory skills are a core requirement in job descriptions for Australian microbiologists (Smith, Grando and Fotinatos 2016), and laboratory accreditation have been emphasized as key learning outcomes for final year microbiology courses (Phillips and Markham 2016). Transferrable problem-solving, planning and organization skills are also highly valued by STEM employers (Rayner and Papakonstantinou 2015), and it is incumbent upon microbiology educators to design assessment activities that align with these desirable employability traits. Practical laboratory examinations represent one form of assessment applicable to this context, which have been well-documented in the literature for medical education. Objective Structured Practical Examinations or Clinical Examinations (OSPE or OSCE) involve individual workstations that students need to visit to be assessed on their clinical skills across a variety of areas under individually supervised examination conditions (Harden and Cairncross 1980). Similar assessment has also been deployed in pharmacy education, where OSCE implementation has been demonstrated to assess a wider set of student competencies as compared to traditional modes of assessment (Kirton and Kravitz 2011). Within science, practical examinations have been used in identifying solution composition in biochemistry (Robyt and White 1990), constructing models of chromosomes during meiosis in genetics (Brown 1990), and apparatus assembly and handling for titrations in chemistry (Kirton, Al-Ahmad and Fergus 2014).

This project describes a practical examination for microbiology—the Laboratory Competency Exam—that has been designed to align with the laboratory skills outlined in the ASM concept-based microbiology curriculum (Merkel 2012). There is an emphasis on tasks that evidence learning outcomes in key laboratory skills, as well as scalability for large class sizes. This Laboratory Competency Exam has been developed in Australia and was presented at the Australian Society for Microbiology Educator's Conference (ASM Educon) in 2014 to a consortium of national leaders in microbiology education. It has since been benchmarked against the national Threshold Learning Outcomes for a Microbiology major in Australia (Burke et al. 2016), and field tested in large classes of up to 400 students per semester. The Laboratory Competency Exam is presented as an assessment instrument that can be combined with existing practical modules to determine student learning outcomes in laboratory skills. Its potential broad applicability will be discussed relative to introductory microbiology courses offered at five Australian institutions with differing educational contexts.

MATERIALS AND METHODS

Assessment format

The Laboratory Competency Exam was deployed at The University of Queensland, Australia from 2016 to 2017 in a second year Microbiology and Immunology course, with up to 400 students enrolled in each offering. Students were informed about the format of the Laboratory Competency Exam at the beginning of the semester and provided with a sample assessment worksheet (Appendix 1) and opportunities to practice the relevant skills. The nine tasks outlined can be completed readily within 60 min, but students were encouraged to consider multitasking to expedite the processes involved.

At the conclusion of the competency assessment, students needed to ensure that all tasks that did not require incubation overnight had been marked on-the-spot by their assessor (Tasks one, two, six, seven and eight). Plates B (Task three—Streak-plating) and C (Task eight—serial dilution: viable plate count), as well as bottles B (Task four—broth culture inoculation), C and D (Task five—sterile broth transfer) all needed to be labeled with student names and identification numbers before being incubated at 37°C overnight.

Teaching assistants supervised and assessed up to six students each within the 60-min timeslot, and approximately 30 min was required to reset the workstations for the next group of students. Sixty minutes should be sufficient time to complete all nine tasks, but more time was allocated to students with relevant medical conditions or accessibility considerations. The marking rubric used by instructors for all nine tasks is outlined in Table1, and the full set of instructions for Faculty is attached in Appendix 2. Once each task was marked, the samples were stored at 4°C and then made available for students to view the following week. Instructors were present to walk students through the results and completed the assessment-feedback loop.

Table 1.

Marking rubric for each of the nine tasks in the Laboratory Competency Exam.

Tasks Fail Pass High Pass
Task 1—Gram-staining Uneven distribution of bacterial cells AND Incorrect Gram-status for stained sample. 0 marks Uneven distribution of bacterial cells OR Incorrect Gram-status for stained sample. 0.5 mark Even distribution of bacterial cells AND Correct Gram-status for stained sample. 1 mark
Task 2—Light microscopy Student could not independently focus on a field of view AND incorrectly identified Gram-status and shape of visualized sample. 0 marks Student could not independently focus on a field of view OR incorrectly identified Gram-status and shape of visualized sample. 0.5 mark Student independently focused on a field of view AND correctly identified Gram-status and shape of visualized sample. 1 mark
Task 3—Streak-plating Absence of any dilution in microbial growth via streaking. 0 marks Successful isolation of single colonies following streaking from the primary inoculum. 1 mark
Task 4—Broth culture inoculation Absence of turbidity in inoculated bottle following incubation. 0 marks Presence of turbidity in inoculated bottle following incubation. 1 mark
Task 5—Sterile broth transfer Presence of turbidity (contamination) in BOTH bottles following incubation. 0 marks Presence of turbidity (contamination) in EITHER bottle following incubation. 0.5 marks BOTH bottles remain sterile following broth transfer and incubation. 1 mark
Task 6—PCR reaction calculations Three or more PCR calculation errors. 0–0.5 marks Two PCR calculation errors. 1 mark At most 1 PCR calculation error. 1.5–2 marks
Task 7—PCR reaction preparation Incorrect final volume AND turbidity in PCR reaction mix. 0 marks Incorrect final volume OR turbidity in PCR reaction mix. 0.5 mark Correct final volume AND turbidity in PCR reaction mix. 1 mark
Task 8—serial dilution: agar plate (viable plate count) Number of colonies on final dilution plate differs by more than an order of magnitude compared to predicted number. 0 marks Number of colonies on final dilution plate within same order of magnitude as predicted number. 1 mark
Task 9—serial dilution: 96-well plate Incorrect 1:10 serial dilution of food dye across 8 wells. 0 marks Correct 1:10 serial dilution of food dye across 8 wells. 1 mark
Total marks out of 10
Tasks Fail Pass High Pass
Task 1—Gram-staining Uneven distribution of bacterial cells AND Incorrect Gram-status for stained sample. 0 marks Uneven distribution of bacterial cells OR Incorrect Gram-status for stained sample. 0.5 mark Even distribution of bacterial cells AND Correct Gram-status for stained sample. 1 mark
Task 2—Light microscopy Student could not independently focus on a field of view AND incorrectly identified Gram-status and shape of visualized sample. 0 marks Student could not independently focus on a field of view OR incorrectly identified Gram-status and shape of visualized sample. 0.5 mark Student independently focused on a field of view AND correctly identified Gram-status and shape of visualized sample. 1 mark
Task 3—Streak-plating Absence of any dilution in microbial growth via streaking. 0 marks Successful isolation of single colonies following streaking from the primary inoculum. 1 mark
Task 4—Broth culture inoculation Absence of turbidity in inoculated bottle following incubation. 0 marks Presence of turbidity in inoculated bottle following incubation. 1 mark
Task 5—Sterile broth transfer Presence of turbidity (contamination) in BOTH bottles following incubation. 0 marks Presence of turbidity (contamination) in EITHER bottle following incubation. 0.5 marks BOTH bottles remain sterile following broth transfer and incubation. 1 mark
Task 6—PCR reaction calculations Three or more PCR calculation errors. 0–0.5 marks Two PCR calculation errors. 1 mark At most 1 PCR calculation error. 1.5–2 marks
Task 7—PCR reaction preparation Incorrect final volume AND turbidity in PCR reaction mix. 0 marks Incorrect final volume OR turbidity in PCR reaction mix. 0.5 mark Correct final volume AND turbidity in PCR reaction mix. 1 mark
Task 8—serial dilution: agar plate (viable plate count) Number of colonies on final dilution plate differs by more than an order of magnitude compared to predicted number. 0 marks Number of colonies on final dilution plate within same order of magnitude as predicted number. 1 mark
Task 9—serial dilution: 96-well plate Incorrect 1:10 serial dilution of food dye across 8 wells. 0 marks Correct 1:10 serial dilution of food dye across 8 wells. 1 mark
Total marks out of 10

Table 1.

Marking rubric for each of the nine tasks in the Laboratory Competency Exam.

Tasks Fail Pass High Pass
Task 1—Gram-staining Uneven distribution of bacterial cells AND Incorrect Gram-status for stained sample. 0 marks Uneven distribution of bacterial cells OR Incorrect Gram-status for stained sample. 0.5 mark Even distribution of bacterial cells AND Correct Gram-status for stained sample. 1 mark
Task 2—Light microscopy Student could not independently focus on a field of view AND incorrectly identified Gram-status and shape of visualized sample. 0 marks Student could not independently focus on a field of view OR incorrectly identified Gram-status and shape of visualized sample. 0.5 mark Student independently focused on a field of view AND correctly identified Gram-status and shape of visualized sample. 1 mark
Task 3—Streak-plating Absence of any dilution in microbial growth via streaking. 0 marks Successful isolation of single colonies following streaking from the primary inoculum. 1 mark
Task 4—Broth culture inoculation Absence of turbidity in inoculated bottle following incubation. 0 marks Presence of turbidity in inoculated bottle following incubation. 1 mark
Task 5—Sterile broth transfer Presence of turbidity (contamination) in BOTH bottles following incubation. 0 marks Presence of turbidity (contamination) in EITHER bottle following incubation. 0.5 marks BOTH bottles remain sterile following broth transfer and incubation. 1 mark
Task 6—PCR reaction calculations Three or more PCR calculation errors. 0–0.5 marks Two PCR calculation errors. 1 mark At most 1 PCR calculation error. 1.5–2 marks
Task 7—PCR reaction preparation Incorrect final volume AND turbidity in PCR reaction mix. 0 marks Incorrect final volume OR turbidity in PCR reaction mix. 0.5 mark Correct final volume AND turbidity in PCR reaction mix. 1 mark
Task 8—serial dilution: agar plate (viable plate count) Number of colonies on final dilution plate differs by more than an order of magnitude compared to predicted number. 0 marks Number of colonies on final dilution plate within same order of magnitude as predicted number. 1 mark
Task 9—serial dilution: 96-well plate Incorrect 1:10 serial dilution of food dye across 8 wells. 0 marks Correct 1:10 serial dilution of food dye across 8 wells. 1 mark
Total marks out of 10
Tasks Fail Pass High Pass
Task 1—Gram-staining Uneven distribution of bacterial cells AND Incorrect Gram-status for stained sample. 0 marks Uneven distribution of bacterial cells OR Incorrect Gram-status for stained sample. 0.5 mark Even distribution of bacterial cells AND Correct Gram-status for stained sample. 1 mark
Task 2—Light microscopy Student could not independently focus on a field of view AND incorrectly identified Gram-status and shape of visualized sample. 0 marks Student could not independently focus on a field of view OR incorrectly identified Gram-status and shape of visualized sample. 0.5 mark Student independently focused on a field of view AND correctly identified Gram-status and shape of visualized sample. 1 mark
Task 3—Streak-plating Absence of any dilution in microbial growth via streaking. 0 marks Successful isolation of single colonies following streaking from the primary inoculum. 1 mark
Task 4—Broth culture inoculation Absence of turbidity in inoculated bottle following incubation. 0 marks Presence of turbidity in inoculated bottle following incubation. 1 mark
Task 5—Sterile broth transfer Presence of turbidity (contamination) in BOTH bottles following incubation. 0 marks Presence of turbidity (contamination) in EITHER bottle following incubation. 0.5 marks BOTH bottles remain sterile following broth transfer and incubation. 1 mark
Task 6—PCR reaction calculations Three or more PCR calculation errors. 0–0.5 marks Two PCR calculation errors. 1 mark At most 1 PCR calculation error. 1.5–2 marks
Task 7—PCR reaction preparation Incorrect final volume AND turbidity in PCR reaction mix. 0 marks Incorrect final volume OR turbidity in PCR reaction mix. 0.5 mark Correct final volume AND turbidity in PCR reaction mix. 1 mark
Task 8—serial dilution: agar plate (viable plate count) Number of colonies on final dilution plate differs by more than an order of magnitude compared to predicted number. 0 marks Number of colonies on final dilution plate within same order of magnitude as predicted number. 1 mark
Task 9—serial dilution: 96-well plate Incorrect 1:10 serial dilution of food dye across 8 wells. 0 marks Correct 1:10 serial dilution of food dye across 8 wells. 1 mark
Total marks out of 10

Reagents and equipment

At the start of the competency assessment, individual stations were set up by laboratory staff to allow each student to complete the assessment within 60 min. Each station is comprised of three Trypticase Soy (TS—15 g/L enzymatic digest of Casein, 5 g/L enzymatic digest of Soybean meal, 5 g/L NaCl) agar plates—Plates A, B, and C. Plates B and C were sterile, while Plate A was inoculated with a pure culture of Staphylococcus aureus (S. aureus) (http://www.amrin.org/CultureCollections.aspx). Each station was also provided with one 5 mL bottle containing TS broth culture of S. aureus (Bottle A), three 5 mL bottles containing sterile peptone water (10 g/L peptone, 5 g/L NaCl) (Bottles B, C, and D) and one 5 mL bottle containing sterile saline (Bottle E). Seven 1.5 mL Eppendorf tubes (Fisher Scientific) were provided to represent reagents for the mock PCR—the tubes for 10× Buffer, MgCl2, dNTPs, F and R primers were filled with Milli-Q water; the tubes for Taq and DNA were filled with red and blue food dye (diluted 1:100 in water), respectively. All plates, bottles and tubes were pre-labeled by the laboratory staff to minimize student confusion during the assessment.

For equipment, students were given two sterile Pasteur pipettes, ten 1.5 mL Eppendorf tubes, P2, P20 and P200 micropipettors and their accompanying pipette tips, one light microscope fitted with a ×100 objective (Olympus), one Gram-staining kit (Becton Dickinson), one Bunsen burner (Labtek), one wire loop, and glass slides. Students were instructed to bring their own safety glasses, laboratory coats and closed-toe footwear to meet the laboratory's Biosafety Level 2 (BSL-2) requirements—gloves, incubation racks and containers, as well as pens for labeling were provided. If the teaching context is restricted to BSL-1, S. aureus can be replaced with Escherichia coli (E. coli) K12 (http://www.amrin.org/CultureCollections.aspx).

Evaluation

Student performance across all nine tasks in the Laboratory Competency Exam was categorized into 'Fail' (<49%), 'Pass' (50%–74%), and 'High Pass' (>75%) grading bands. Survey questions ranking student confidence in laboratory skills assessed in the nine tasks were quantified using a five-point learning-gains scale (1 = Do not know how to do; 2 = Not competent; 3 = Need Practice; 4 = Competent; 5 = Highly Competent). Students were invited to voluntarily complete the survey after completing the Laboratory Competency Exam in both 2016 and 2017, with a response rate of 12% and 17%, respectively. Informed consent for student participants was sought in accordance with The University of Queensland's Institutional Review Board ethics approval for research involving human participants (Project Number 2012000755).

RESULTS

The Laboratory Competency Exam is a tool that can be affixed to the end of a learning sequence to evaluate its effectiveness in developing student laboratory skills in microbiology. This activity is intended for introductory microbiology courses, and readily applicable for students pursuing majors in microbiology, biology and biotechnology within the first 2 years of their undergraduate study in science. It assesses students on their individual laboratory competencies and works well in small class sizes; however, it has been field-tested in courses with large student cohorts (up to 400 students per semester) supported by teaching assistants.

Students should have completed introductory biology courses where bacterial cell structure and function and the central dogma of biology have been covered as core concepts. They were also required to complete BSL-2 training prior to attending laboratory classes in microbiology. The learning sequence to be evaluated through the deployment of the Laboratory Competency Exam should cover Gram-staining, light microscopy, aseptic technique, pipetting, serial dilution and dilution calculations. The assessment itself requires two separate sessions (approximately 3 h each) to setup individual student stations, incubate samples and provide feedback viewing sessions for students once the marks have been finalized. This does not include the prior learning time and opportunities provided for students to develop these practical competencies, which may vary depending on the learning objectives of the course. A minimum learning time of 2–3-h laboratory sessions for students to develop laboratory competencies is recommended prior to the exam.

Learning objectives

The Laboratory Competency Exam assesses if students are able to:

  1. Presumptively identify bacterial samples through Gram-staining and light microscopy.

  2. Use aseptic culturing techniques to safely isolate and culture microorganisms.

  3. Quantify the number of microorganisms in a sample using serial dilution and viable plate count techniques.

  4. Prepare solutions and reaction mixes through accurate calculations, measurements, and pipetting.

These learning objectives were developed in close alignment with the ASM concept-based microbiology curriculum, in particular the key competencies in laboratory skills (Merkel 2012). They do not focus on documentation and reporting on experimental protocols, allowing instructors to tailor communication-orientated assessment to their individual contexts. The tasks for this activity have also been cross-referenced against Australian guidelines, where there is significant overlap between these learning objectives and the national Australian Threshold Learning Outcomes for a Microbiology major (Burke et al. 2016). This information is summarized in Table2.

Table 2.

Alignment between the Laboratory Competency Exam, ASM Key Laboratory Competencies and Australian Threshold Learning outcomes in microbiology.

ASM key laboratory competencies Australian threshold learning outcomes for a microbiology major Module learning objective Laboratory competency exam task
Properly prepare and view specimens for examination using microscopy 2.2: Demonstrating competency in core microbiological skills and techniques3.2: Designing and planning a safe and efficient investigation or experiment.3.3: Selecting and applying relevant and appropriate practical and/or theoretical techniques or tools 1, 2 Tasks 1 and 2
Use pure culture and selective techniques to enrich for and isolate microorganisms 2 Tasks 1, 3, 4, 5, and 6
Use appropriate methods to identify microorganisms 1, 2 Tasks 1 and 2
Estimate the number of microorganisms in a sample 2, 3 Tasks 8 and 9
Use appropriate microbiological and molecular lab equipment and methods 1, 2, 3, 4 Tasks 1–9
Practice safe microbiology, using appropriate protective and emergency procedures 5.1: Working effectively, responsibly, and safely with microorganisms 2 Tasks 1–6
Document and report on experimental protocols, results, and conclusions 3.4: Collecting, accurately recording, interpreting, and drawing conclusions from scientific data4.1: Using appropriate written and oral forms to communicate understanding of microbiology to a broad range of stakeholders Not assessed directly.
ASM key laboratory competencies Australian threshold learning outcomes for a microbiology major Module learning objective Laboratory competency exam task
Properly prepare and view specimens for examination using microscopy 2.2: Demonstrating competency in core microbiological skills and techniques3.2: Designing and planning a safe and efficient investigation or experiment.3.3: Selecting and applying relevant and appropriate practical and/or theoretical techniques or tools 1, 2 Tasks 1 and 2
Use pure culture and selective techniques to enrich for and isolate microorganisms 2 Tasks 1, 3, 4, 5, and 6
Use appropriate methods to identify microorganisms 1, 2 Tasks 1 and 2
Estimate the number of microorganisms in a sample 2, 3 Tasks 8 and 9
Use appropriate microbiological and molecular lab equipment and methods 1, 2, 3, 4 Tasks 1–9
Practice safe microbiology, using appropriate protective and emergency procedures 5.1: Working effectively, responsibly, and safely with microorganisms 2 Tasks 1–6
Document and report on experimental protocols, results, and conclusions 3.4: Collecting, accurately recording, interpreting, and drawing conclusions from scientific data4.1: Using appropriate written and oral forms to communicate understanding of microbiology to a broad range of stakeholders Not assessed directly.

Table 2.

Alignment between the Laboratory Competency Exam, ASM Key Laboratory Competencies and Australian Threshold Learning outcomes in microbiology.

ASM key laboratory competencies Australian threshold learning outcomes for a microbiology major Module learning objective Laboratory competency exam task
Properly prepare and view specimens for examination using microscopy 2.2: Demonstrating competency in core microbiological skills and techniques3.2: Designing and planning a safe and efficient investigation or experiment.3.3: Selecting and applying relevant and appropriate practical and/or theoretical techniques or tools 1, 2 Tasks 1 and 2
Use pure culture and selective techniques to enrich for and isolate microorganisms 2 Tasks 1, 3, 4, 5, and 6
Use appropriate methods to identify microorganisms 1, 2 Tasks 1 and 2
Estimate the number of microorganisms in a sample 2, 3 Tasks 8 and 9
Use appropriate microbiological and molecular lab equipment and methods 1, 2, 3, 4 Tasks 1–9
Practice safe microbiology, using appropriate protective and emergency procedures 5.1: Working effectively, responsibly, and safely with microorganisms 2 Tasks 1–6
Document and report on experimental protocols, results, and conclusions 3.4: Collecting, accurately recording, interpreting, and drawing conclusions from scientific data4.1: Using appropriate written and oral forms to communicate understanding of microbiology to a broad range of stakeholders Not assessed directly.
ASM key laboratory competencies Australian threshold learning outcomes for a microbiology major Module learning objective Laboratory competency exam task
Properly prepare and view specimens for examination using microscopy 2.2: Demonstrating competency in core microbiological skills and techniques3.2: Designing and planning a safe and efficient investigation or experiment.3.3: Selecting and applying relevant and appropriate practical and/or theoretical techniques or tools 1, 2 Tasks 1 and 2
Use pure culture and selective techniques to enrich for and isolate microorganisms 2 Tasks 1, 3, 4, 5, and 6
Use appropriate methods to identify microorganisms 1, 2 Tasks 1 and 2
Estimate the number of microorganisms in a sample 2, 3 Tasks 8 and 9
Use appropriate microbiological and molecular lab equipment and methods 1, 2, 3, 4 Tasks 1–9
Practice safe microbiology, using appropriate protective and emergency procedures 5.1: Working effectively, responsibly, and safely with microorganisms 2 Tasks 1–6
Document and report on experimental protocols, results, and conclusions 3.4: Collecting, accurately recording, interpreting, and drawing conclusions from scientific data4.1: Using appropriate written and oral forms to communicate understanding of microbiology to a broad range of stakeholders Not assessed directly.

Tasks one and two revolve around using Gram-staining and light microscopy to presumptively identify a bacterial sample inoculated on Plate A. Once students have completed their Gram-stain and focused on a field of view using the ×100 objective of the light microscope, they are instructed to notify their assessor. The instructor then grades the quality of the Gram-stain by cell density, color and the students' ability to correctly identify the Gram-status and cell morphology of the specimen. Staphylococcus aureus should be identified by students as Gram-positive cocci.

Tasks three, four and five rely on pure culture and aseptic technique, and students are expected to inoculate agar plates using the streak plating dilution technique, and transfer broth culture and sterile broth with Pasteur pipettes. Tasks six and seven involve PCR calculations and preparing a mock PCR. The reagents in the mock PCR are all comprised of water with the exception of Taq (red food coloring) and DNA (blue food coloring). Given the small volumes required of these two reagents and the resulting mixture of two different food dye colors with water, the final reaction mastermix should produce a unique color that is easily discernible by eye. Instructors can then verify the accuracy of the pipetting by both the final volume in the tube, as well as the color/optical density of the reaction mix.

Tasks eight and nine require students to conduct serial dilutions in two different scenarios. Task eight involves 10-fold serial dilutions of S. aureus broth culture, with students only plating out the final dilution. Following incubation overnight at 37°C, the number of colonies present on this plate should closely match the plates prepared by laboratory staff. This visual readout is indicative of students' competencies in serial dilution and estimating microbial numbers in a sample using viable plate count methodology. Task nine involves diluting food-coloring across eight wells in a 96-well plate, and the progressive dilution in color intensity across the wells can be verified by the assessor as confirmation of dilution and pipetting accuracy. A plate reader can be used to measure the optical density for each well, but any dilution in the intensity of food-coloring is also easily detectable by eye.

Field testing

To determine the effectiveness of the Laboratory Competency Exam in large classes, the learning activity was implemented within the 2016 and 2017 offerings of an introductory microbiology module offered at The University of Queensland (UQ), Australia. Leading up to the Laboratory Competency Exam, students completed 4 weeks of laboratory classes as part of the previously described Oral Microbiome project (Wang et al. 2015)—387 and 300 second year students were recruited for the project in 2016 and 2017 respectively. The Laboratory Competency Exam was designed to be summative in nature, accounting for 10% of the course grade. Accordingly, the assessment is administered close to the end of the semester, and students are given multiple opportunities to refine these skills in previous laboratory classes. The time limit and examination conditions are important for the individual evaluation of competencies for each student, and to prepare them for the processing of large numbers of samples in short timespans—a common predicament in diagnostic and research laboratories facing outbreaks, increased surveillance requirements, or complex experimental setups.

To compare student attempts at Gram-staining against established standards, we referred to ASM's online image gallery for Gram-stained samples of microorganisms (http://asmscience.org/content/education/imagegalleries). The most common student mistakes across the nine tasks in the Laboratory Competency Exam are highlighted in Fig.1, which compares student-generated samples against those prepared by instructors. In Task three, students often forgot to flame the wire-loop in between sets of four streaks for streak-plating, leading to an inability to dilute the primary inoculum and visualize single bacterial colonies (Fig.1A). In task eight, students frequently failed to dilute the broth culture as part of their 10-fold serial dilutions, leading to a much higher number of colonies in their final dilution plate as compared to those prepared by instructors (Fig.1B). The degree of turbidity expected for broth culture inoculation and sterile broth transfer in tasks four and five can be seen in Fig.1C and D, respectively.

Figure 1.

Sample student data in the Laboratory Competency Exam. (A) Task 3 (Streak-plating) and (B) Task 8 (Serial dilution—viable plate count): student samples are presented on the left, indicative of 'Fail' grades for the specified tasks. Instructor samples are shown on the right. (C) Task 4 (broth culture inoculation): broth culture of S. aureus is used to inoculate sterile peptone water and incubated at 37°C for 24 h. Samples are assessed on turbidity—the tube shown above represents an accurate result. (D) Task 5 (Sterile broth transfer): sterile broth is transferred from the original bottle to the transfer bottle using a Pasteur pipette, then incubated 37°C for 24 h. No visible turbidity should be observed in either bottle as shown.

Sample student data in the Laboratory Competency Exam. (A) Task 3 (Streak-plating) and (B) Task 8 (Serial dilution—viable plate count): student samples are presented on the left, indicative of 'Fail' grades for the specified tasks. Instructor samples are shown on the right. (C) Task 4 (broth culture inoculation): broth culture of S. aureus is used to inoculate sterile peptone water and incubated at 37°C for 24 h. Samples are assessed on turbidity—the tube shown above represents an accurate result. (D) Task 5 (Sterile broth transfer): sterile broth is transferred from the original bottle to the transfer bottle using a Pasteur pipette, then incubated 37°C for 24 h. No visible turbidity should be observed in either bottle as shown.

Evidence of student performance across the laboratory competency tasks is outlined in Fig.2. A significant portion of students struggled with streak-plating (Task three) with 13%–35% failing the task across 2016–2017. Serial dilution using the viable plate count method (Task nine) also posed a challenge to a lesser extent for many students, with 20%–23% failing the task across the 2 years of implementation. Generally, the students fared well in the other tasks though, with >80% of the cohort scoring a High Pass in the rest of the skills between 2016 and 2017, and >90% of the cohort scoring a High Pass for the Laboratory Competency Exam overall (Fig.2). These results are corroborated by post assessment learning-gains surveys, where >50% of the student population across 2016 and 2017 expressed that they were 'Confident' or 'Highly Confident' in light microscopy, Gram-staining, pure culture, viable plate count, pipetting, and dilution calculations (Fig.3). Laboratory skills involving 'microbial identification' and 'planning my own experiment' scored slightly lower with only 40% of the cohort responding with 'Confident' or 'Highly Confident', but this may reflect the additional complexity involved in these higher order laboratory competencies that incorporate multiple techniques. This data collectively suggest that participation in the Oral Microbiome project (Wang et al. 2015) over 4 wk is sufficient training for students to develop key laboratory competencies in microbiology.

Figure 2.

Student performance in laboratory tasks in the Laboratory Competency Exam. The proportion of students in 2016 (n = 387) and 2017 (n = 300) who achieved a Fail (<49%—white), Pass (50%–74%—grey), or a High Pass (75%–100%—black) within each of the tasks is shown above. Each task was graded using the marking rubric outlined in Table 1.

Student performance in laboratory tasks in the Laboratory Competency Exam. The proportion of students in 2016 (n = 387) and 2017 (n = 300) who achieved a Fail (<49%—white), Pass (50%–74%—grey), or a High Pass (75%–100%—black) within each of the tasks is shown above. Each task was graded using the marking rubric outlined in Table1.

Figure 3.

Student confidence in laboratory skills following the Laboratory Competency Exam. Student confidence in each laboratory skill was measured through voluntary student responses to survey questions following the completion of the Laboratory Competency Exam in 2016 (n = 45/387, 12% response rate) and 2017 (n = 51/300, 17% response rate) on a 1–5 scale (1 = Do not know how to do; 2 = Not competent; 3 = Need Practice; 4 = Competent; 5 = Highly Competent). The percentage of student responses falling within each category of confidence is shown above.

Student confidence in laboratory skills following the Laboratory Competency Exam. Student confidence in each laboratory skill was measured through voluntary student responses to survey questions following the completion of the Laboratory Competency Exam in 2016 (n = 45/387, 12% response rate) and 2017 (n = 51/300, 17% response rate) on a 1–5 scale (1 = Do not know how to do; 2 = Not competent; 3 = Need Practice; 4 = Competent; 5 = Highly Competent). The percentage of student responses falling within each category of confidence is shown above.

DISCUSSION

The combination of performance and survey data outlined above indicates that the Laboratory Competency Exam is able to measure whether or not a learning sequence can fulfill key learning objectives in microbiology laboratory skills. In describing this assessment activity for the first time, we have established close alignment between the tasks, learning objectives, ASM's Laboratory Competencies (Merkel 2012) and the Australian Threshold Learning Outcomes for a Microbiology major (Burke et al. 2016) (Table2), to further support its validity as a diagnostic tool for measuring student competencies. The pedagogical value of assessment is governed by its applicability across different educational contexts and to assist instructors with adopting the laboratory competency exam, and to date the assessment has only been deployed at The University of Queensland (UQ). We will describe the factors involved in practical assessment in institutions across four Australian states, to compare and contrast the applicability of the Laboratory Competency Exam in different contexts: UQ in Queensland, Monash University and The University of Melbourne in Victoria, University of Technology Sydney (UTS) in New South Wales, and Edith Cowan University (ECU) in Western Australia.

Program structure

Looking across the Australian program offerings in microbiology, UQ, Monash University, The University of Melbourne, UTS and ECU are all research-intensive universities with class sizes ranging from 200 to 1200 students per year. Microbiology is typically offered as a study option in the second year of 3-year undergraduate programs in Science and Biomedical Science (Table3). This learning sequence provides students with opportunities to develop basic laboratory skills and prerequisite biological knowledge in the first year of their undergraduate studies, before entering a microbiology laboratory in second year. Class size is a core consideration, as large class size tends to reduce the breadth and depth of learning outcomes while hampering the capacity to provide individualized student feedback (Cuseo 2007). In our experience, however, laboratory skills assessment is scalable given appropriate teaching assistant support, typically at 1:12 instructor to student ratios; hands-on laboratory skills assessment is a compulsory assessment item in four out of five of these Australian institutions, and the Laboratory Competency Exam was field-tested at UQ with class sizes approaching 400 students a year.

Table 3.

Overview of teaching modes in introductory microbiology subjects across five Australian Higher Education Institutions.

Institution and course Modes of instruction Student numbers Lab hours Laboratory skills covered Assessment
UQ: 'Microbiology & Immunology' Lectures and laboratory practicals (in-person) ∼400 second year students 30 h Light microscopy, Gram-staining, aseptic technique, microbial identification, serial dilution, solution and dilution calculations Laboratory notebooks, written reports, hands-on skills assessment, intra and end-of-semester exams.
Monash: 'Introduction to Microbiology & Microbial Biotechnology', 'Microbes in Health & Disease'. Lectures and laboratory practicals (in-person and online) ∼300 second year students 36 h Light microscopy, Gram-staining, aseptic technique, microbial culture, microbial identification (biochemical and molecular), serial dilution, dilution calculations Reports, quizzes, hands-skills assessment (laboratory skills tests), teaching associate assessment, intra and end-of-semester exams
The University of Melbourne: 'Molecular & Cellular Biomedicine', 'Principles of Microbiology & Immunology' Lectures (including flipped classroom sessions) and laboratory practicals (in-person) ∼1200 second year students 21 h Aseptic technique, streak plate dilution, Gram-staining, Light microscopy, microbial identification (biochemical and molecular), antimicrobial susceptibility Quizzes (online and in-person), intra- and end-of-semester exams.
UTS: General Microbiology, Epidemiology & Public Health Microbiology, Clinical Bacteriology Lectures, laboratory practicals, flipped classroom sessions, writing workshops ∼550 second year students 31 h Light microscopy, Gram-staining, aseptic technique, serial dilution and calculations, growth curve and generation time calculation, microbial identification, microbial isolation, media and nutrition Online quizzes, written assignments, hands-on skill tests, intra- and end-of-semester exams
ECU: Applied Microbiology Lectures and laboratory classes (in person), tutorials, lectures and laboratory classes (in person) ∼200 second year students 40 h Light microscopy, aseptic technique, staining, bacterial identification, Laboratory notebooks, hands-on assessment, online MCQ, intra- and end-of-semester exams
Institution and course Modes of instruction Student numbers Lab hours Laboratory skills covered Assessment
UQ: 'Microbiology & Immunology' Lectures and laboratory practicals (in-person) ∼400 second year students 30 h Light microscopy, Gram-staining, aseptic technique, microbial identification, serial dilution, solution and dilution calculations Laboratory notebooks, written reports, hands-on skills assessment, intra and end-of-semester exams.
Monash: 'Introduction to Microbiology & Microbial Biotechnology', 'Microbes in Health & Disease'. Lectures and laboratory practicals (in-person and online) ∼300 second year students 36 h Light microscopy, Gram-staining, aseptic technique, microbial culture, microbial identification (biochemical and molecular), serial dilution, dilution calculations Reports, quizzes, hands-skills assessment (laboratory skills tests), teaching associate assessment, intra and end-of-semester exams
The University of Melbourne: 'Molecular & Cellular Biomedicine', 'Principles of Microbiology & Immunology' Lectures (including flipped classroom sessions) and laboratory practicals (in-person) ∼1200 second year students 21 h Aseptic technique, streak plate dilution, Gram-staining, Light microscopy, microbial identification (biochemical and molecular), antimicrobial susceptibility Quizzes (online and in-person), intra- and end-of-semester exams.
UTS: General Microbiology, Epidemiology & Public Health Microbiology, Clinical Bacteriology Lectures, laboratory practicals, flipped classroom sessions, writing workshops ∼550 second year students 31 h Light microscopy, Gram-staining, aseptic technique, serial dilution and calculations, growth curve and generation time calculation, microbial identification, microbial isolation, media and nutrition Online quizzes, written assignments, hands-on skill tests, intra- and end-of-semester exams
ECU: Applied Microbiology Lectures and laboratory classes (in person), tutorials, lectures and laboratory classes (in person) ∼200 second year students 40 h Light microscopy, aseptic technique, staining, bacterial identification, Laboratory notebooks, hands-on assessment, online MCQ, intra- and end-of-semester exams

Table 3.

Overview of teaching modes in introductory microbiology subjects across five Australian Higher Education Institutions.

Institution and course Modes of instruction Student numbers Lab hours Laboratory skills covered Assessment
UQ: 'Microbiology & Immunology' Lectures and laboratory practicals (in-person) ∼400 second year students 30 h Light microscopy, Gram-staining, aseptic technique, microbial identification, serial dilution, solution and dilution calculations Laboratory notebooks, written reports, hands-on skills assessment, intra and end-of-semester exams.
Monash: 'Introduction to Microbiology & Microbial Biotechnology', 'Microbes in Health & Disease'. Lectures and laboratory practicals (in-person and online) ∼300 second year students 36 h Light microscopy, Gram-staining, aseptic technique, microbial culture, microbial identification (biochemical and molecular), serial dilution, dilution calculations Reports, quizzes, hands-skills assessment (laboratory skills tests), teaching associate assessment, intra and end-of-semester exams
The University of Melbourne: 'Molecular & Cellular Biomedicine', 'Principles of Microbiology & Immunology' Lectures (including flipped classroom sessions) and laboratory practicals (in-person) ∼1200 second year students 21 h Aseptic technique, streak plate dilution, Gram-staining, Light microscopy, microbial identification (biochemical and molecular), antimicrobial susceptibility Quizzes (online and in-person), intra- and end-of-semester exams.
UTS: General Microbiology, Epidemiology & Public Health Microbiology, Clinical Bacteriology Lectures, laboratory practicals, flipped classroom sessions, writing workshops ∼550 second year students 31 h Light microscopy, Gram-staining, aseptic technique, serial dilution and calculations, growth curve and generation time calculation, microbial identification, microbial isolation, media and nutrition Online quizzes, written assignments, hands-on skill tests, intra- and end-of-semester exams
ECU: Applied Microbiology Lectures and laboratory classes (in person), tutorials, lectures and laboratory classes (in person) ∼200 second year students 40 h Light microscopy, aseptic technique, staining, bacterial identification, Laboratory notebooks, hands-on assessment, online MCQ, intra- and end-of-semester exams
Institution and course Modes of instruction Student numbers Lab hours Laboratory skills covered Assessment
UQ: 'Microbiology & Immunology' Lectures and laboratory practicals (in-person) ∼400 second year students 30 h Light microscopy, Gram-staining, aseptic technique, microbial identification, serial dilution, solution and dilution calculations Laboratory notebooks, written reports, hands-on skills assessment, intra and end-of-semester exams.
Monash: 'Introduction to Microbiology & Microbial Biotechnology', 'Microbes in Health & Disease'. Lectures and laboratory practicals (in-person and online) ∼300 second year students 36 h Light microscopy, Gram-staining, aseptic technique, microbial culture, microbial identification (biochemical and molecular), serial dilution, dilution calculations Reports, quizzes, hands-skills assessment (laboratory skills tests), teaching associate assessment, intra and end-of-semester exams
The University of Melbourne: 'Molecular & Cellular Biomedicine', 'Principles of Microbiology & Immunology' Lectures (including flipped classroom sessions) and laboratory practicals (in-person) ∼1200 second year students 21 h Aseptic technique, streak plate dilution, Gram-staining, Light microscopy, microbial identification (biochemical and molecular), antimicrobial susceptibility Quizzes (online and in-person), intra- and end-of-semester exams.
UTS: General Microbiology, Epidemiology & Public Health Microbiology, Clinical Bacteriology Lectures, laboratory practicals, flipped classroom sessions, writing workshops ∼550 second year students 31 h Light microscopy, Gram-staining, aseptic technique, serial dilution and calculations, growth curve and generation time calculation, microbial identification, microbial isolation, media and nutrition Online quizzes, written assignments, hands-on skill tests, intra- and end-of-semester exams
ECU: Applied Microbiology Lectures and laboratory classes (in person), tutorials, lectures and laboratory classes (in person) ∼200 second year students 40 h Light microscopy, aseptic technique, staining, bacterial identification, Laboratory notebooks, hands-on assessment, online MCQ, intra- and end-of-semester exams

Modes of instruction

Close alignment between lectures and accompanying practical sessions in the laboratory represents the primary mode of instruction across the five Australian institutions in this study. There remains a strong emphasis on laboratory skills in each institution—laboratory contact per semester ranges from 21 to 40 h for every student, with wide-ranging practical exercises designed to engage students providing real-life context to the techniques and skills being taught and developed. Assessment of laboratory activities and topics range from multiple-choice quizzes, laboratory notebook submissions, scientific report write-ups of laboratory results, hands-on skills assessment and short-answer questions relating to laboratory topics in written exams. There is clear consistency across the five institutions in the modes of delivery, contact hours and the types of laboratory assessment utilized (summarized in Table3).

Laboratory activities

UQ's research-based laboratory exercises in the oral microbiome have been previously documented (Wang et al. 2015), but different laboratory learning activities are equally amenable to the implementation of the Laboratory Competency Exam. At Monash University, the practical classes include the identification of a bacterial isolate from mock patient samples, water and food quality testing and monitoring the spread of antibiotic resistance. All activities are designed to develop core microbiological skills and competencies including aseptic technique, microscopy, culturing (both from the environment and the human body), identification (including the use of molecular techniques such as PCR) and other fundamental skills such as viable plate count method to enumerate bacterial concentration. Streak-plating, Gram-staining, light microscopy, serial dilution and viable plate count are all assessed in a hands-on skills assessment.

The Department of Microbiology and Immunology at the University of Melbourne offers scenario-based investigations of mock disease outbreaks. Students are provided with a number of swab samples taken from the different patients involved, and various locations and items near the outbreak site. Students complete a series of basic microbiological techniques, including aseptic technique and streak plate dilution, Gram staining and light microscopy. Bacteriological, virologic and immunological topics are further investigated using additional techniques including viable counts, PCR and agarose gel electrophoresis, enzyme immune-assays, and hemagglutination inhibition assays—all designed to presumptively identify and characterize the causative agent.

Microbiology at UTS is focused on the development and reinforcement of practical skills and practice-based learning. There are a number of laboratory-based threshold skills assessments that are considered to be critical to progression, and Gram-staining and microscopy in particular are repeated a number of times across several learning scenarios. A practical skills test in a General Microbiology subject has been implemented in a manner consistent with the design philosophy of the Laboratory Competency Exam described above. This skills test includes drawing growth curves and generation time calculations, completing serial dilution calculations, setting up light microscopes to Kohler illumination, completing a Gram reaction of two unknown bacteria, and determining Gram reaction, cell morphology and cell arrangement.

ECU's Medical Microbiology practical classes have a strong medical identification focus. Classes progressively introduce bacteria by Gram stain profile and then by sample type (wound swab, cerebral spinal fluid, blood culture), with additional parasitology, fungal and virus-based practical classes. For the concepts of microbial identification, students perform staining and basic biochemical tests in the early weeks followed by case histories including antimicrobial therapy in the final two weeks. There is a mixed paper and practical assessment at the end of the semester, with the practical tests focusing on streak plating, Gram staining and microscopy.

Evidently despite slight variations in instructional modes and learning sequences, all five Australian institutions focus on practical laboratory skills in direct alignment with the learning activities, objectives, threshold learning concepts and core competencies outlined by ASM and the Australian national guidelines (Merkel 2012; Burke et al. 2016), and in turn the Laboratory Competency Exam described in this project. The individualized exam-based setting of these assessment tasks have the potential to differentiate student outcomes across different learning activities (Suits 2004), which can be useful for instructors looking to evidence the learning gains from new laboratory modules (Shortlidge and Brownell 2016; Wang 2017). This speaks to the broad applicability of this assessment item, and its potential to add to the growing body of pedagogical resources available for microbiology educators (Merkel 2016).

Future directions

The practical skills that are emphasized across the ASM concept-based curriculum as well as the Laboratory Competency Exam are largely focused on visualizing culture-dependent laboratory techniques. Biochemistry and molecular biology techniques are also a core component of the modern microbiologist's toolkit but have not yet been incorporated into skills assessment in large classes. Given the modest cost and wide availability of PCR reagents and machines, laboratory exercises involved in recombinant DNA technology can be considered as the next competency to be scaffolded into threshold skills assessment; many such exercises are well-documented in the literature (Robertson and Phillips 2008; Wang et al. 2012; Hargadon 2016). This can also be closely coupled to the use of bioinformatics tools, where short in silico competency tasks can be designed in alignment with key learning outcomes in bioinformatics education (Furge et al. 2009). This project describes a prototypical version of Laboratory Competency Assessment for microbiologists, which can be readily expanded upon into different areas of specialization. In the short term, we hope to expand the Laboratory Competency Exam beyond UQ into all five Australian institutions described; our long-term vision is for instructors to use and adapt the assessment tools described, and benchmark microbiology practical standards for graduates and prospective employers.

Acknowledgements

The authors would like to acknowledge the contributions of the following academic and laboratory preparation staff involved in conceiving and developing laboratory practical sessions at each of our institutions: Mr Mohamed Mohideen and the invaluable help of the Microbiology laboratory preparation staff; Dr Kylie Wilson, Dr Fiona Glenister, Ms Shelly Thirwell and Ms Danielle West (Monash); Ms Sandra Uren, Ms Cheryl Power, Ms Helen Cain, Prof. Roy Robins-Browne, Ms Rita Infelice, Ms Christine Alaan, Ms Justine Tippet, Ms Gina Holland, Ms Wendy Larrad, and Ms Eleonora Puglia (University of Melbourne); A/Prof Maurizio Labbate, and all of the academic and professional team members involved in microbiology class teaching and preparation for General Microbiology and other Microbiology classes (UTS); Dr Kuo-Chang Lee, Mr Igor Popovic, Mr Peter Kraat for the preparation and design of the Laboratory Competency Exam, as well as all of our graduate teaching assistants (UQ).

Conflict of interest. None declared.

REFERENCES

Brown

C

.

Some misconceptions in meiosis shown by students responding to an advanced level practical examination question in biology

.

J Biol Educ

1990

;

24

:

182

6

.

Burke

C

,

Cain

H

,

Coleman

N

et al.

Threshold learning outcomes for a microbiology major

.

Microbiol Aust

2016

;

37

:

93

7

.

Cuseo

J

.

The empirical case against large class size: adverse effects on the teaching, learning, and retention of first-year students

.

J Faculty Dev

2007

;

21

:

5

21

.

Furge

LL

,

Stevens-Truss

R

,

Moore

DB

et al.

Vertical and horizontal integration of bioinformatics education

.

Biochem Mol Biol Educ

2009

;

37

:

26

36

.

Harden

RM

,

Cairncross

RG

.

Assessment of practical skills: the objective structured practical examination (OSPE)

.

Stud Higher Educ

1980

;

5

:

187

96

.

Hargadon

KM

.

A model system for the study of gene expression in the undergraduate laboratory

.

Biochem Mol Biol Educ

2016

;

44

:

397

404

.

Kirton

SB

,

Al-Ahmad

A

,

Fergus

S

.

Using structured chemistry examinations (SChemEs) as an assessment method to improve undergraduate students' generic, practical, and laboratory-based skills

.

J Chem Educ

2014

;

91

:

648

54

.

Kirton

SB

,

Kravitz

L

.

Objective structured clinical examinations (OSCEs) compared with traditional assessment methods

.

Am J Pharm Educ

2011

;

75

:

111

.

Merkel

S

.

The development of curricular guidelines for introductory microbiology that focus on understanding

.

J Microbiol Biol Educ

2012

;

13

:

32

38

.

Merkel

SM

.

American Society for Microbiology resources in support of an evidence-based approach to teaching microbiology

.

FEMS Microbiol Lett

2016

;

363

:

fnw172

.

Paustian

TD

,

Briggs

AG

,

Brennan

RE

et al.

Development, validation, and application of the Microbiology Concept Inventory

.

J Microbiol Biol Educ

2017

;

18

:

pii: 18.3.49

.

Phillips

M

,

Markham

J

.

Providing an authentic experience of laboratory accreditation processes in a final year microbiology unit

.

Microbiol Australia

2016

;

37

:

90

2

.

Rayner

G

,

Papakonstantinou

T

.

Employer perspectives of the current and future value of STEM graduate skills and attributes: an Australian study

.

J Teaching Learning Graduate Employability

2015

;

6

:

100

15

.

Robertson

AL

,

Phillips

AR

.

Integrating PCR theory and bioinformatics into a research-oriented primer design exercise

.

CBE Life Sci Educ

2008

;

7

:

89

95

.

Robyt

JF

,

White

BJ

.

Laboratory practical exams in the biochemistry lab course

.

J Chem Educ

1990

;

67

:

600

.

Rybarczyk

BJ

,

Walton

KL

,

Grillo

WH

.

The development and implementation of an instrument to assess students' data analysis skills in molecular biology

.

J Microbiol Biol Educ

2014

;

15

:

259

67

.

Seitz

HM

,

Horak

RE

,

Howard

MW

et al.

Development and validation of the microbiology for health sciences concept inventory

.

J Microbiol Biol Educ

2017

;

18

:

pii: 18.3.54

.

Shapiro

C

,

Moberg-Parker

J

,

Toma

S

et al.

Comparing the impact of course-based and apprentice-based research experiences in a life science laboratory curriculum

.

J Microbiol Biol Educ

2015

;

16

:

186

97

.

Shortlidge

EE

,

Brownell

SE

.

How to assess your CURE: a practical guide for instructors of course-based undergraduate research experiences

.

J Microbiol Biol Educ

2016

;

17

:

399

408

.

Smith

JV

,

Grando

D

,

Fotinatos

N

.

Graduate employment trends in the life sciences: implications for microbiology educators

.

Microbiol Aust

2016

;

37

:

56

9

.

Suits

JP

.

Assessing investigative skill development in inquiry-based and traditional college science laboratory courses

.

Sch Sci Math

2004

;

104

:

248

57

.

Wang

JT

.

Course-based undergraduate research experiences in molecular biosciences—patterns, trends, and faculty support

.

FEMS Microbiol Lett

2017

;

364

:

1

9

.

Wang

JT

,

Daly

JN

,

Willner

DL

et al.

Do you kiss your mother with that mouth? An authentic large-scale undergraduate research experience in mapping the human oral microbiome

.

J Microbiol Biol Educ

2015

;

16

:

50

60

.

Wang

JTH

,

Schembri

MA

,

Ramakrishna

M

et al.

Immersing undergraduate students in the research experience

.

Biochem Mol Biol Educ

2012

;

40

:

37

45

.

Supplementary data