Are there particular standards for ethical recommendations? I wonder because ethics is sometimes a subjective and/or cultural idea.

Seems important

How do novice moderators get experience then?

Is it possible and effective to do testing via something like Skype or Zoom? It's not really mentioned here, but it could be useful in today's world.

How would one classify a person that may not have used the specific product, but who regularly uses similar issues? As an example, suppose someone has not done the Tourette's class before, but they have participated in similar classes for CEUs.

In some situations, perhaps culturally or otherwise, some people may want to appear to be agreeable? How can we best ask this question without biasing the responses?

Here's the science and art thing coming up again.

What's the history / reasoning for the change?

This can definitely be true for designers. I'm also wondering about users who may be in a semi-design / semi-user capacity such as teachers and instructors. Does empathy work the same in those situations?

An important point and one that is not easily subsituted.

Is memory and./or unconscious thought ever an issue? For example, do learners sometimes not know why they fixated on certain things?

So it seems it is less than expert review, but what makes heuristic review less biased?

Is bias an issue in heuristic evaluation?

I agree it is important not to be leading, but sometimes also, there is data that we really want to find. Is part of not being leading while getting data that is needed a subtle power of suggestion while maintaining neutrality (as much as possible) (e.g., asking what works, if anything, so that there's balance between what is effective and what is more problematic)?

Can we assume, as with qualitative case studies, that this individual must be chosen carefully? If so, what are some good things to look for in choosing a representative individual? Does it vary in terms of the prototype or product?

This sounds like a useful feature.

So basically it is useful, but we want to limit spending a lot of time on it because it doesn't have the functionality.

Would be interested in hearing how PowerPoint is useful since, unlike the specialized software, it's not as designed for that purpose.

One would have to be somewhat aware of how the digital space would function, though, I would think before functionality could be assessed in its entirety.

Feedback needed to promote a better quality learner experience

Is this possibly a disincentive toward higher fidelity prototyping? At the very least, it would seem to me to suggest that one really wants to ensure one has things relatively right before doing a higher level prototype.

Which requires some amount of vigilance.

Is the purpose just to look for problems? How about looking for things that are effective? Does looking for problems bias against looking for effectiveness? If so, how does one create a balance or, at least, provide a space where both problems and effectiveness can be examined?

Does this possibly indicate the possibility of bias?

Each technology has to consider how it works for various groups.

Seems to me that this is designing for the learners to make a shift, but not necessarily know they are doing it. I wonder, though, if it is supposed to be unconscious, how it can be critiqued or measured for effectiveness. Is it simply a manner of whether the intended unconscious adjustment works or not?

Can these be considered both at the same time? They are two different groups, yet perhaps we can learn about learners through teacher experience for example (I'm thinking especially about younger, elementary age children who we may not get as much data from. Still, such an approach would seem to have some limitations in that the teacher cannot necessarily know everything about the user's experience as that is personal to them. It's also possible that teachers might project their own feelings / experiences onto students (e.g. Hofmann et al., 2020).

Which points out the limitations of the original tools

Applied and focused is important in terms of keywords. First, it rejects / diminishes the more theoretical and then also states the need for focus, so less of a free ranging exploration.

Sounds like interesting is a risk (but couldn't being uninterested also be a risk?).

Are there some guidelines for what this might be?

Looks like this is similar to what we have in a lot of our UF branding.

Considering this goes to Harvard's colors, it's very much of advantage of them that this carousel worked like that so the harmony could exist.

I'd like to know more what is meant by comical. Would this exclude intentional humor?

Interesting that these things are similar more universally despite the cultural differences.

This makes the UX designer especially important in the process. It seems kind of instructor / designer heavy though. Could this work as well in a constructivist or constructionist environment?

WebAIM is good and all, but for best practice, it's really good to use several tools and maybe even a manual check / testing with actual disabled users as well. Not specifically aimed at WebAim. It's more like automatic tools and simulators in general.

Could this also change over time? I know there have been some Western connotations of purple that came from the fact that purple dye was once rare and now that it's more common, there's more access.

Good thing we at UF are more blue than orange then (more seriously, how about when they're mixed?)!.

Keeping working is important, though, because sometimes people have motivations, but don't always put in the work.

The co-founder of UDL, David Rose, who I had for a course during my M.Ed. program, always expressed the importance of emotion in learning. Attitudes and interests were definitely part of that, so I'm fascinated by how color impacts those things.

Super interesting!

Would make a great demo for a class.

Speaks to what I was saying in the other article.

Ineffective could relate to accessibility too!

The corporate can be less flexible, but I wonder about other institutions. For instance, universities are very brand oriented I'd argue these days.

Systematic is a key word.

This makes a lot of sense. We need to be mindful in the moment and accept feedback from others; it's (hopefully) not personal.

Self as judge. For what it's worth, I do think if one wouldn't personally like the product, that's telling that others may not be happy with it either.

Yes. Helpful to orient the more verbal thinking individuals.

Who decides what is unimportant or distracting though? People may have some variability in what they pay attention to (e.g. some people on the autistic spectrum or have other nuerodivergences - I'm thinking especially of Temple Grandin's work with animal enclosures) and, in fact, it could be what is dismissed as "unimportant" or even not noticed by one or even most people could end up actually being very important. I do think there's some onus on the designer or communicator to explain though in a way that points the detail out to others in a clear manner.

I wonder if there would be a possibility of bias for these community-based tools. Not everyone uses such sites or has access.

I was thinking about this. Does this have any impact on the color ideas in the bullet points though.

Curious to know what the difference between "somewhat" and "extra" is.

Yes, indicating that someone has to be very visually gifted in order to understand or needs to have a certain competency can be limiting. The design should be flexible enough for all levels and abilities.

I think this supports the idea that negative isn't always bad for instructional design.

Being able to see how something looks like can help the learner "do" a bit more.

Which makes it more challenging maybe.

Considering brain plasticity, would this be true also for people with visual impairments or whose visual perception skills may be limited compared with what's normative?

How do we bring culture into the idea of building visuals? I'm thinking especially of pluralistic societies or pluralistic events such as international conferences.

Super important. When visuals are too hard to process (or too busy) it makes what is being communicated less clear and the viewer is likely going to really struggle to know what is going on.

Is pleasing / displeasing measurable (like is there some sort of scale)? It could be subjective potentially, so I'm wondering how we can know pleasing / displeasing ahead of time, especially in situations where the learner is not directly known to us.

How does visual attention work for people who have visual impairments or who may have non-normative visual processing? For those individuals, I would argue they are going to attend to the textual information over the visual. I'm not arguing necessarily that visual information would be totally immune, but would it be possible to "paint" through words (e.g. alt text).

It seems to me like negative isn't necessary "bad" for holding an "invitation" (although not saying negative is always okay either) as long as it gets attention and invites the viewer into the product.

I can see this being true for adults, but is this true for children as well (especially very young children). Their interests can fluctuate a little more.

So it fluctuates. When attention and interest are a little lower, how does one get it back? Can it be recovered?

Our initial impressions form how we process the rest of the experience.

Clear visuals (that maybe can also be explained) can be really important for people with visual processing (not related to visual impairment necessarily more like a learning disability or neurological type thing) issues for exactly these reasons.

I wonder if this could include options for senses. If one sense is diminished, would emotional connection from an equivalent option be able to take over?

Perception can refer to the senses. My work in accessibility discusses perception through the senses as part of the POUR (perceivable, operable, understandable, robust) principles, but perhaps connection could also be made to emotion. POUR doesn't currently call out emotion directly, but maybe there would be potential.

Importance of emotion. I often wonder about emotion in learning from a usability / accessibility perspective. It's so much of a product (how people connect with it).

Sort of a way of saying it is a science and an art.

Which indicates a risk and may also not be compatible with certain personality factors some people might have.

Communication does take up a lot of time especially if one is trying to get things right.

Why?

Does this get modified if someone is unable to stand? If only that person sits, is this exclusionary?

Sometimes, arguably, it takes a bit of work for this to happen. Also, is simplicity compatible with quality?

Would this still be the case?

What does one do with demotivated individuals? Are we building for motivation?

Is this always known?

Such as?

Is complexity level always known from the start?

How does one get stakeholders to begin observing progress? If not voluntary from the start, how does one incentivize stakeholders to do this?

?

Stresses need to be assertive "nicely". Does this rely on some privileges or perceptions, though? Also, contexts can be different, so, if it is possible to train for assertiveness in these situations, how might one go about doing that?

This makes some sense because it is needed in project management, but it doesn't map to a particular kind of technology or project, so can be used across contexts.

Again, generalist knowledge is needed.

This could get overwhelming, though. How much is too much?

We need to detangle, to get to what people really seem to want.

This would seem to suggest that constantly cooperating with others is needed as a sounding board or reviewer. Universities often have writing and communication centers for their students, but this continued support does not always exist past one's student days and into professional life. Yet some peer support or feedback unless there is somebody like a writer / editor / communications advisor employed for this purpose. This seems to propose the project manager take on some of these roles.

Calls for smarts, but also humility.

In my M.Ed. program, we were required to do internships. I would suppose some working on real world design problems like we are doing in this class is useful as well. How much, though, does internships and "real word" academic experience count, though, versus more longer term paid employment?

I wonder, for the faculty members in the study, what they would think about this. Where would they go? I assume the SME. Would there be some assumption or even less humble thinking that they are the SME, though, even maybe a little? If so, how does one approach other SMEs or how does that impact the relationship?

Interesting perspective. While there are constraints to being under-resourced, these resources often bring their own restrictions which can limit innovation.

Working knowledge if not expertise

Wider themes seem to have more agreement.

A lot of responses in this category and not a lot of agreement.

Need for experiential experiences

Again, another tie in between PM and ID.

Takes some finesse to get it right.

Importance of the context

The similarities can be used to connect PM with ID.

Particular kinds of communications; it isn't a vague competency, but a targeted one.

Could suggest divergence in agreement, but could also be indicative of a large amount of resources.

This seems key, but it may be influenced by particular organizational cultures, personal exposure, and so on.

The bigger the organization, the more specialization is needed and/or desired.

Such specialists could be harder to come by in a smaller organization though.

Can be a bigger fish in a smaller pond.

Could be some gaps between the PMP and actual practice, so gaps between what is taught and what is practiced isn't necessarily just limited to universities.

It's a risk, though, instead of a gaurentee.

They don't say whom, but I wouldn't be surprised if the faculty members were the ones making up most of these five.

I guess the academic role also counts within the criteria.

The representation of this figure is interesting. I'm interpreting it to be three intersecting Venn diagrams and I guess the KSAs are coming out of the intersections.

PM does seem very quantitative oriented to me so am not surprised by this.

Another academic / practice divide. How can education technology type programs work better with what is needed in practice? Especially for more practice oriented degrees, this matters. Moreover, in so far as PM- prepared IDs are needed in higher ed, higher ed institutions would seem to be undermining their own institutions if they are not offering this knowledge.

Is this a valid assumption?

I notice that sometimes things that are done a lot in practice (or have a great deal of interest in practice) are not always reflected in research or academic interest.

Hence the emphasis in the project management course.

I had not thought of using AI for instructional design although for some models I suppose it is possible.

With building to fail, I would think it would be more about an openness to failure or acceptance of failure. Thought and effort should be put into the prototype, but it is necessary to accept that things will not necessarily work and that's okay.

Manipulatives seem like a way to make sketching / storyboarding not entirely visual, which allows for greater participation.

Can our own self-perceptions really be put aside or do we simply manage them? It's definitely important to look for even small things that, in the end, actually make an outsize difference as the Good Kitchen example shows.

The last element they list "share the story" wasn't really talked about much in the other readings so far, but it does show how we communicate about design adds to others' perceptions about how good it is. I'm thinking a bit about the late Steve Jobs with this. He was pretty influential in his presentation of designs to the public; if he had lacked that talent, something like an iPhone may not necessarily have caught on like it did. Jobs created buzz.

Two things seemed really significant here to me: divergence (to create ideas) and convergence (to agree upon ideas). The second idea is that it is supposed to be a way to do design for non-designers.

This one's also good.

I want to show my appreciation for this title. It's creative / memorable.

I know that experience made a big difference for some colleges / universities and perhaps K-12 schools in 2020 during the COVID situation. Those institutions who already had some online infrastructure had somewhat of an advantage over institutions who had little or no experience with online learning (that's not to say every institution that didn't have online flopped or that everyplace that did have online infrastructure was successful, but in most cases, I think having some previous experience was extremely helpful. I also observed that some of the job ads for instructional designers temporarily shifted to more experienced people in Spring and Summer 2020.

Having a sort of common understanding is helpful on teams. This indicates that people are less likely to be confused about what to do.

Who or what determines what the facts are? How do the "facts and results" account for the theories that are driving what the facts are?

So several things might happen at once. They are not necessarily dependent on each other to be completed before moving on.

Seems like a broad statement.

Important that they note prototyping is a team effort.

This is essentially telling how they would design the idea if they were in charge.

To them, I imagine it is more like bodystorming.

Showing in a situation

The active nature of bodystorming is something I would connect to needing to know more about contexts.

Sounds like we can maybe learn certain things from bodystorming that we can't necessarily learn from other design practices.

This sounds fantastic to me! Personally, I like methods that are a little more involved.

Building is risky, but not as risky as doing something more "final" without knowing if it's right or not. Product builds can, then, maybe be thought of as more of a risk management strategy.

When doing something unfamiliar, sometimes we don't know how it will turn out unless we take the risk of building / creating; these risks help us learn.

I've noticed in the project management course that a lot of the tools sort of tell a visual story of what is going on. I suppose this could be another tool.

Despite the fact that storyboarding is drawing from the visual spatial, some of these ideas of sequencing and flow are also applicable to the writing process. Could really vivid, detailed writing substitute for sketching. With this idea, I'm also thinking about people who may be limited in terms of drawing and visual skills because of a physical or learning / cognitive sort of disability. How do they design or are they excluded?

Non-verbal is tied to the visual.

Suggestive that the visual spatial is important on design.

Something like a sketch is good for imagining physical space or a physical object if one doesn't actually have access yet.

I find this a little confusing. Is the sample not actual users?

With the emphasis on visual ideas, to me, this questions some of the effectiveness of low fidelity prototypes.

This is clarifying about the differences between low and high fidelity prototyping.

Importance of visualization in prototyping

Interesting finding. Would be curious to know more about how this works.

Iteration allows us to build on what we have overlooked.

Arguably, architecture is art.

Art doesn't have to be useful, but I think it is quite a far reaching statement to imply that no art is useful. Writing, music, or visual art, for instance, might be meant to provoke to make something happen for instance.

Suggestive that one should take time to generate a lot of ideas. Ideas could come together and lead to better ideas.

Noticing the similarities between this and computer science education. Karen Brennan, who I worked with during my M.Ed. program, and her lab have a whole "getting unstuck" curriculum.

Arguably also the industrialized influences on education. Also indicative that fast does not account for quality; for that, being steady, but not fast, would be better.

Could be one of the reasons the group was having issues.

I wish examples had been provided here.

Another execution issue for this group

This is a good point; sometimes more inclusive sounding design ideas don't always deliver on that inclusion in practicality / delivery if not in principle. I'm not sure it is an indictment of the idea itself, but could point to the importance of execution.

Takes also some confidence in one's abilities.

Seems to be a lack of resources, but on the other hand, also a lack of creativity / insight stemming from the status quo / being too comfortable with an idea.

I think we've seen that already somewhat in our recent design project we just completed.

That's why it would be helpful to build rapport over time, but also to demonstrate trust in how the data is used and handled.

I think it may be applicable what the age of these end-users are.

Also applicable to design based research and other forms of research.

Interesting that the analytic and creative are applied like this. How much does analysis have a role in creativity and the reverse?

That's what I associate the most with design thinking.

I like the idea of early insights. It seems to me like sometimes we aren't aware of insights or at least how they play out until the end of the project. Probably both are useful although their formulation and how in-depth those insights are can vary.

Prevailing things continue sometimes even if they are shown to be flawed.

Usability is not just related to using a product; it also relates to how it is used which may be related to social norms.

Broadening understanding

Rules would have to be somewhat institutionalized, though, and not all learning places are super institutional. Could be a barrier in informal learning for instance.

Activity theory broadens what is being examined.

How would vicarious memories be viewed outside certain settings, though? For instance, I'm not satisfied that an employer would be satisfied with vicarious memories. Could be situational.

Simply personas without a story and maybe some of the more humanistic or nuanced information?

LXD and society

Others point out potential issues of representation which may be valued, but at least empathy interviews are a way of discussing with a real person so it's not creating without any input.

That does give us a lot more detailed information that could be very helpful.

Not just the learning theory, but learning theory applied to a particular situation.

Technology can shape how people make sense of the world; there may be an argument as to how positive this is.

How well would this work for informal as opposed to formal learning?

Technology use should be considered within a broader framework or broader context.

Draws on qualitative type theory

Helps to organize information about the learner

Seems logical enough

Culture and ability status (among other things) seem to be ignored in this particular section unlike other demographic data.

Seems maybe a little contradictory given the emphasis on the majority of learners. At minimum, it would imply universal features are more about differentiation than something designed for every learner.

I agree that factual information is best. However, it is also true in society that facts are becoming more clouded or controversial. This is something LXDs are going to have to think about in the future.

How much of an issue are varied codes of conduct on ethics? Are any contradictory?

Nothing on disability interestingly enough unless that is supposed to be categorized here. That would be a bad way of including disability, though, if it is IMO as not all disability accessibility things are solved with tech alone.

We assume that personas are important enough that we aren't going to get it completely right the first time. It's more of a process than a one-off sort of thing.

Learning to appreciate constraints is something we can take from persona building and something personas can really enhance.

I found it interesting with Jamie Ann's persona that they basically describe someone with ADHD or some similar type executive processing issue, but never say that outright. That could be trying to make the persona more general.

I had a creative writing background prior to switching to education. This reminds me of workshopping.

How much of an issue is this outside an artificial type environment where people get to pick personas? What about the ones "not like us?"

Experience building alongside the academic.Needs to be authentic or else it is still merely an academic exercise.

Interestingly, even with the degree, they still think they need to grow their skills. Definitely could be "lifelong learning" but also could be indicative of what some ID-related programs teach.

Interesting that there is so much emphasis on building skills and portfolios and what not, but not on instructional design for its own sake. Maybe interest in instructional design was read as interest in theory?

Ideas seem to get started by reflecting on the self.

So this definitely takes some co-construction of narrative here, but it is also important that the designers aren't necessarily co-constructing here as there is no actual person. Others have mentioned the potential for bias and here is a place where that could possibly happen.