Please see the comments in the quiz.

Add the full directions for Q&A discussions.

I think it'd make sense to use a page instead of an assignment. In the long term, it'd be best to get a better scan.

Please see the comments in the assignment.

The article is mentioned in the directions. However, it doesn't appear to be "included in the assignment." On a related note, providing a link in multiple locations can be helpful, but it can also lead to discrepancies and other errors.

This is helpful guidance. The assignment questions act as guiding questions.

I see that the readings below relate to this topic. Perhaps they will help. Consider mentioning them here.

Link to the free article on the web. Use the title for the link. https://www.healthaffairs.org/doi/10.1377/hlthaff.2017.0218

Be sure to use unique descriptive text for links. It'd be better to use the topics. You could include the pages, too. (This comment applies to multiple links on this page.)

Example: part about the size of social loss and its determinants (pp. 206-213)

Are you referring to the video titled "Moral Hazard and Social Loss"? If so, I recommend saying that.

Is it possible that other instructors will use this course content in the future? If so, I recommend using third-person language here.

Consider adding an outcome about defining moral hazard. (Same goes for the previous module and adverse selection.)

This is another good third-party video that illustrates the multimedia principle.

Multiple-select question

Nice application and comprehension questions

Good comprehension questions. I noticed a minor typo. Above the abstract, the subheading is "Question 2" instead of "Part 2."

I recommend providing info about the task and point distribution on the Canvas page. See the Week 1 Assignment.

This module is a good one, overall. The content, especially the examples, and the practice exercise are highlights. I see connections between the module components, including the outcomes and assessments (though these connections could be strengthened).

Link: https://www.nber.org/system/files/chapters/c9822/c9822.pdf

Link out to the web or a library database, when possible. This is a best practice promoted by the library.

Consider rewording (e.g., using a different phrase than "correct... answers"). On the one hand, nearly all instructors want students to provide accurate information and to construct logical arguments. On the other hand, many instructors also want students to demonstrate creativity, innovation, etc.

Please see my comments in the discussion.

Nice mix of comprehension and application questions. Consider adding explanations for the T/F questions. (I see that students are encouraged to "post a question in the Week 4 Q&A if [they] need an explanation for any of those answers.")

Rename to "Practice Exercise" (as in previous modules).

Good third-party video. I appreciate the simple visuals (stick people, numbers, etc.) and clear explanations. This video illustrates Mayer's multimedia principle: People learn more from words and relevant images than words alone. The video is also captioned.

Good examples of solutions to the adverse selection problem.

Consider removing the Conclusion slide. It feels a little out of place. Also, its text is small, and some text is located at the bottom where captions would appear.

For me, the first half was clear and helpful, but the second half was less so. On the Correlation slide, what do the horizontal lines represent?

I recommend using third-person language if other instructors will use this course content in the future. In this particular context, it's clear that "I" refers to the author of the video (not anyone teaching the course).

Good thinking. These examples are in-depth but not overly long.

Good content. These authentic examples will help students understand the concept of adverse selection. Four should be sufficient.

Be consistent about quiz settings. Do you want to always show correct answers immediately? The first two quizzes have a different setting.

Compared to the practice exercise, the quiz seems to emphasize comprehension more than application. The quiz seems fine though.

Multiple-select question

If you stick with this prompt, it would be a good idea to mention the article title ("The Importance of Relative Prices in Health Care Spending") and module (Week 2).

Be sure to update the discussion settings.

I could be mistaken, but this prompt seems to only indirectly connect with the outcomes. If I'm on target, I recommend considering how to make the connection more direct. That could involve revising or replacing the prompt.

This is a good module, overall. I see the connections between the module components, including the outcomes and assessments.

The RAND video and New England Journal of Medicine article are examples of high-quality third-party content. They both have accessibility issues, however. The article's issues can be easily addressed by linking to a library database article.

Consider adding a community-building assignment. We recommend including one in every module. However, it's OK to occasionally make an exception -- for example, due to high workload.

Please see my comments in the quiz.

Nice use of signaling (circling) when discussing the study results and the table. It may have been a good idea to add a study slide before the table. No big deal though.

This Q&A discussion has a different prompt.

Important admonition

Link to the article on the web. https://www.trillianthealth.com/company/blog/winning-a-losing-game-part-vi-the-impact-of-price-elasticity

Please see my comments in the assignment.

Link to the ProQuest article. https://xavier.idm.oclc.org/login?url=https://www.proquest.com/scholarly-journals/oregon-experiment-effects-medicaid-on-clinical/docview/1348130590/se-2?accountid=407

Nice tip

Convert the preceding title into a link. Unfortunately, this video doesn't have captions (neither at the RAND website nor on YouTube).

In an email, you wrote, "Objectives for week 3 are set up as questions. I like the change in format as I think it may draw students in." I'm unsure of student opinion, but I think a regular outcome is more precise. The verb provides information about what students should be able to do. It also indicates the level of the outcome. Instructors and instructional designers often distinguish between lower-level and higher-level outcomes.

Good mix of comprehension, application, and reflection questions. The questions also seem to connect with the outcomes.

Three T/F questions

Do you want to show correct answers immediately? Just checking. The previous quizzes have different settings. Showing correct answers immediately can facilitate learning, but some instructors have concerns about academic integrity.

Remove this broken link. Avoid linking to the main pages. The URL of each main page is based on the page title, which contains dates that vary from semester to semester. When the title changes, the URL changes, and the link breaks.

Seven questions. Consider adding info about how the points are distributed.

Good comprehension questions. I noticed a few potential issues: * Q4 refers to "Bhattacharya, Hyde, and Tu (2014) textbook on price elasticity of demand." Where is this reading? * Q7 begins by referring to Table 3, but Q7C refers to Table 2.

Good module, overall. I see alignment between the module components (outcomes, content, practice activities, graded assignments). The practice activities and the corresponding lightboard videos represent an excellent opportunity for students. They seem to be more comprehensive than the graded assignments. In this module and future ones, try to ensure that graded assignments target all of the outcomes.

Consider doing the following: * Clarify the directions for the Week 2 Q&A. Copy and paste the directions into other Q&A discussions. * Add discussion questions to the Week 2 Discussion. Try to target one or two outcomes (e.g., welfare economics and public health policy). * Order captions for Dr. France Weaver's videos. Do you see her videos in Vidgrid? Paul may need to order the captions. * Add guiding questions and/or reflection questions relating to each video and reading.

Please see my comments in the quiz.

Make sure that these due dates match the table at the top of the page.

These outcomes are well-written overall, but they could be improved. Here are some pointers:

• Outcome 2: Each outcome typically has one verb. This facilitates assessment.
• Outcome 3: Consider using the term "explain."
• Outcome 4: Consider using a more concrete term.

Please see my comments in the discussion board.

Link to article.

Other strategies may be more effective than highlighting. I previously mentioned the acronym PICSS (paraphrase, infer, connect, self-question, summarize).

These handwritten notes should be converted into digital text at some point.

9:59

Could the Word document be converted to Canvas content? Do students benefit from the Word format?

Could the Word document be converted to Canvas content? When I try to preview the document, I get an error message. I was able to download and view the document though.

Good note. It should reduce student confusion. See my comment above about colorization.

Nice practice activity. I see application and comprehension questions. Students will appreciate the lightboard videos with answer explanations.

Page 118 is missing.

Consider using guiding questions instead. Also, see my comment above about colorization.

Colorized text should also be italicized. This is an accessibility issue. That said, it's a good idea to keep colorization to a minimum and to use colors consistently.

Please see my comments in the discussion board.

This paragraph and the one below appear in Week 1's Overview. They appear in the Time Management section here. Again, try to be consistent.

The Week 2 Discussion has separate due dates for the post and reply. The Week 1 Discussion doesn't. I recommend being consistent with due dates (and terminology).

Nice application questions

Remove this broken link. Avoid linking to the main pages (e.g., Week 1, Week 2, etc.). The URL of each main page is based on the page title, which contains dates that vary from semester to semester. When the title changes, the URL changes, and the link breaks.

Be sure to add a discussion prompt and remove this language. If the discussion is set up properly, students won't be able to post here. See the following article: How do I create a group discussion in a course?

What is the percentage value of Q&A discussions? What is the percentage value of other discussions? Complete setup under the Assignments tab.

The criteria are still vague. Do students need to comment a minimum number of times on separate days?

This module is strong in terms of content. I think it could be improved in terms of assessment, organization, and quality control. The module components have a medium-level of alignment.

Consider doing the following: 1. Use your revised outcomes. Also, add a few outcomes (e.g., positive vs. normative statements, externalities, etc.). 2. Add quiz questions connected to the outcomes. 3. Use days of the week instead of dates. Add them to all of the assignment and activity sections of the main page. 4. Add guiding questions and/or reflection questions below the video tables. The former highlights important takeaways. The latter encourages students to make different types of connections.

When possible, link to articles instead of uploading them to Canvas. This is a best practice relating to copyright.

Sunday

Write "Go to the Week 1 Practice Quiz." Convert the assignment title into a link.

Good open-ended question. It should spur discussion.

What if "Sarah" doesn't post early in the week? I recommend asking everyone to answer the question instead of using the "hot seat" approach. Make sure to activate the "Users must post before seeing replies" feature. If you want more intimate discussions, you can use group discussions instead of class discussions.

Where's the discussion forum? Add a link.

If you want students to post in the Week 1 Q&A (not recommended), provide a link.

Use more specific descriptive text.

Where's the due date for the infographic activity?

Use descriptive text for links. Write: "Go to the M1 'Who Pays' Activity" (or something similar).

This ungraded activity is categorized as a graded assignment.

Use more specific descriptive text (e.g., video title).

Do other faculty members own this video and the videos below? If so, we should look into what happens if a faculty member leaves, retires, etc. I assume that the videos remain on the VidGrid server.

Check the links. This table should have four unique links: two PowerPoint links and two video links.

Consider removing this sentence if you want students to answer other students' questions as part of their grade.

This time estimate seems low to me.

Make sure that this info matches the Due Dates table. Also, use days of the week instead of dates.

This is inaccurate. Correct or remove.

Is this the correct discussion? If not, fix the link.

I recommend using the revised outcomes that you wrote during IOCD. (I've pasted them below. I made a few minor adjustments.) You could add a few more (e.g., positive vs. normative statements, externalities, etc.).

1. Explain what the field of health economics focuses on.
2. Describe the various commodities of healthcare.
3. Describe U.S. healthcare spending over time.
4. Compare U.S. healthcare spending to other developed countries.
5. Define economic market failures.
6. Identify market failures in the healthcare sector.

Consider using the same headings as in the online course template. (The heading is Overview. The subheadings are Introduction, Outcomes, and Due Dates. You could add a Time Management subheading.) The idea is for online students to have a somewhat consistent experience across courses.

If you want to keep the current module design (Weeks 1-8 in one container and Weeks 9-16 in another), place the week dates in the page titles. Just be aware that each page's URL is based on the page title. Therefore, you shouldn't create any links to the main pages (e.g., hypothetical link from Week 1 Q&A to Week 1: Introduction and Context). This precaution will prevent broken links in the future.

Questions 1-4 seem reasonable. They emphasize recall and comprehension. However, the questions don't seem to have a strong connection to the module outcomes. I recommend targeting the outcomes first and then branching outward.

What's the rationale for Question 5? I see that it's tied to the icebreaker.

How do you track these participation/engagement grades? How do you provide transparency for students?

Unfortunately, it's not possible to annotate the graph. Another option is a discussion forum. Each assignment type has upsides and downsides.

Consider providing more options to spur discussion. For example, tell the students to post at least two comments (inferences, connections, questions, or replies).

Add your questions to the Canvas assignment and the Hypothesis assignment. Copy from Canvas to Hypothesis.

I recommend placing the introductory paragraph on the module main page and in the assignment. Place the assignment details in the assignment alone.

Make sure that this grading info matches the assignment's point value.

I see three discussion forums. I recommend combining them. In other words, ask students to complete all three parts/phases in the same place. Add directions under the Task heading.

Be sure that your text and rubric match your Assignments tab. Also, consider updating the rubric. It is somewhat vague, and it has multiple references that seem more appropriate for an in-person class (e.g., "unnotified absence").

In your experience, how well do the weekly Q&A discussion forums work? I see that students can comment or reply to another student if they don't have any questions.

Remove the dates from the title of the discussion forum. That way, you won't need to update them every semester. Instructors often forget to make some updates, resulting in student confusion.

Consider using a Canvas rubric. Also, make sure the rubric points match the assignment points.

Nice assignment. I like the time adjustment and the question options. Students appreciate options, and two or three is a good number. Too many options can be counter-productive.

Consider using headings to make the directions easier to navigate and process, as you did in the discussion.

Nice prompts. Both are personalized. Both relate to the same topic. However, they approach the topic from slightly different angles.

I recognize this text from IOCD! Feel free to borrow.

Good thinking. One minute is too short in my experience. Three minutes seems more appropriate.

• Good start. Overall, the module is well-organized, and the progression of instructional content makes sense.
• I see connections between the SLOs and the learning activities and assignments.
• Nice job of dividing the discussion directions into parts. I encourage you to do something similar with other assignments. Highly organized content is easier for students to navigate and process.
• Make sure that the course content is accessible (e.g., links, images, tables, etc.).
• Check the settings of the discussion and other assignments.

I can help with the HTML/CSS if you'd like. The substance of the page is the most important thing, but the styling can be helpful for signaling and accessibility.

Good introduction. It is clear and concise, and it provides a preview of what is to come.

This video only has auto-generated captions. It is not accessible for students with deafness. Videos with human-generated captions are typically much more accurate.

Consider adding this video to the previous page.

This conclusion gets the job done. It's clear and concise. You could potentially elaborate a bit on the first sentence, but that's your decision.

This video is accessible for students with deafness but not blindness. If you get a student with blindness, you can provide an accommodation.

It's a best practice to include the length.

Example: video (3:22)

Good introduction. Consider adding some guiding questions or reflection questions, as you've done elsewhere.

Consider limiting this part to a discussion of the activity from the instructor's perspective, and moving the student directions to the section below.

As I wrote in my feedback on the M4 Discussion in IOCD, "I agree that this prompt has a 'low floor' and 'high ceiling.' The prompt emphasizes observation, creative thinking, and multiple perspectives. You even state, 'There are no wrong answers.' I like how you require students to explain their reasoning when they answer the ... question about which tag doesn't belong."

Would you ask middle grades students to do this, too? Just wondering.

Nice explanation of the purpose of this assignment

Nice reflection question. Consider highlighting such questions using styling.

This page is good in terms of substance. Reviewing the standards and reflecting upon them are valuable activities. You've provided helpful introductory and contextual info, too. However, the page has a lot of accessibility issues.

Nice use of conversational language, here and below

I commend you for including the corresponding SLO. Do you plan to do this for all assignments?

As I wrote in my feedback on the CSW in IOCD, "Good idea for a self-assessment. Consider asking students to explain in 1-2 sentences why they believe they need more assistance with each standard." Also, consider using headings to make the directions easier to navigate and process, as you did in the discussion.

These links have good descriptive text.

Check the assignment settings. Do you want the assignment to be graded? Do you want the discussion to be threaded? Do you want students to have to post before they can see others' posts?

The content of the module seems to be high quality. It appears to primarily relate to the journal article.

The module would benefit from some editing though. See my recommendations throughout.

Decide whether to add a quiz.

Is this a low estimate? The estimated time for the M3 Case Study is 4-6 hours.

This table mentions a quiz, but the quiz hasn't been built and published.

I changed the due dates of the two readings from Wednesday and Friday to your standard ones of Monday and Wednesday. Having early due dates is important for a short work week.

I added this link to a library resource. Note the link's prefix (https://xavier.idm.oclc.org/login?url=).

I created corresponding reading assignments in the Assignments area.

Add a few more outcomes (e.g., defining and distinguishing between reckoning and judgment).

This part feels a bit redundant. Consider referring to the inventions of the computer, Internet, and World Wide Web.

I already proofread and edited this page.

Change the title to "Model 3: Reckoning and Judgment Mutually Assist Each Other."

Identify the section (Scenario C).

Add an Example: Wearing Health Monitors heading above this paragraph.

Also, consider adding two subheadings: Description and Discussion.

Consider elaborating a bit on this topic.

Change the title to "Model 2: Reckoning Serves Judgment."

Add an Example: Piloting Military Drones heading above this paragraph.

Also, consider adding two subheadings: Description and Discussion.

Identify the section (Scenario B).

Change the page title to "Model 1: Reckoning Replaces Human Judgement."

Identify the section (Scenario A).

Add an Example: Selling Credit Cards heading above this paragraph.

Also, consider adding two subheadings: Description and Discussion.

Consider removing this part and simply describing the AI system in this paragraph.

Consider elaborating and explaining the meaning of some of these terms.

It may be a good idea to present the definition of reckoning first (same order as in the example). This isn't that important though.

Add a line break and a Definitions heading.

Also, use the word "module."

Is this judgment? If so, say that.

Use the same approach as with the previous example. Describe the situation. Then, explain the type of decision making.

Add an Example of Reckoning vs. Judgment heading above this paragraph.

Consider reorganizing the info a bit. For example, you could begin by writing: "An example can help us differentiate between two types of decision making: reckoning and judgment. Suppose you are faced with a choice..."

I see that the directions are the same as in the previous case study.

Write M6 Case Study: Google (as in M3 Case Study: Wells Fargo).

Add an Introduction heading.

Interesting scenario. It seems relevant to the content presented in the module, and it certainly raises ethical questions.

What if students were asked to apply the models first and then briefly discuss the moral issues of each approach? They could also be asked to select their favored approach and to explain their thinking.

Add an Image Types heading above this paragraph.

Add a conclusion and additional resources.

Which pages/sections?

Add an Implications heading above this paragraph.

Add an Introduction heading above this paragraph.

Introduction heading

Types of Design heading

Introduction heading

Move this content to the previous page. Add an Immoral Consequences section.

Add a Computer Algorithms heading above this paragraph.

I recommend saying something about computers and judgment in this section. You could simply mention that the idea will be explored on the pages that follow.

Reword this part.

Add an Introduction heading above this paragraph.

In this case, there is a correct or right answer, so does it make sense to put these words in quotes?

Is the choice about the answer or the method to reach the answer?

Introduction heading