As your R^2 value will always increase as you add more predictors, the adjusted R^2 value attempts to take this into account and give a more accurate representation of the relationship. Adjusted R^2 is also used more in prediction when lots of data points and variables are being used.

Overall, this article has an appearance of organization and good statistical protocols but, in my opinion, lacks in true depth and exploration of the variables. This doesn't mean there aren't valuable findings and take-aways - no article is perfect!

A lot of these statements at the end are very general and not grounded in particular research findings from this study. Perhaps a deeper assessment from the results of "cognitive engagement" that can give practitioners tangible advice on how to design flipped learning activities for students.

I'm not completely sure what they mean by this. Do they wish they had operationalized this concept differently? Another sentence would help to clarify.

Limitation #3: Other variables that may influence student outcomes in a flipped learning environment

Limitation #2: "perceived" outcomes by self-reported survey may not be as strong as other measures.

Limitation #1: Not generalizable outside of this setting (common limitation in research)

This statement seems irrelevant without elaboration or relation to the research questions directly.

Acknowledgement of other variables that may impact the outcome but that weren't in the scope of this study. Always good to see this was thought about and to bring to the reader's attention.

It's important that researchers identify their limitations when discussing results. This is an important statement.

"Prior knowledge levels" are not necessarily reflected in GPA. It's important to keep assertions grounded in the way variables were operationalized.

I still wish we had clear parameters of what "high", "mid", and "low" GPAs were for this study.

This writes up in long form what Figure 2 shows in diagram.

A variety of model fit tests were run. While each test has parameters for level of fit, there are not set rules when looking at the tests as a whole. Here all tests showed a good fit so not an issue but be aware that there may be times that some tests say a model is a good fit and some say it is not - then, if it's your research, you need to make a decision and clearly justify based on the tests.

Finding: (1) academic capability (GPA) -> content/learning-related outcomes AND (2) epistemological beliefs (constructivist) -> content/learning-related content

Finding: academic capability (GPA) -> Engagement (pre-class & in-class)

Finding: academic capability (GPA) -> learner engagement (overall score) -> learning outcomes

The beta value here demonstrates how much the outcome variable (total engagement) will increase based on 1 unit change in the predictor (in-class engagement).

This is the R^2 value from Table 4.

Always pay attention to what the asterisks mean by looking at the table notes.

This is the effect size (eta squared). Refer to the benchmarks given earlier on what makes small, medium, large. The authors use Cohen's benchmarks which are commonly used.

Remember: measured as numerical values from the survey

Remember: measured as categories (high/medium/low)

Just a reminder that ANOVAs need categorical independent variables (hence the high/mid/low categorization here) and continuous dependent variables (they are using numbers from the surveys).

I always look for 3 "quick stats" when reading research. 1) the test statistic - here it's the F statistic that includes degrees of freedom 2) the p value which tells me if there is statistical significance and 3) the effect size which tells me the weight of the results. While they don't list all three here, they do in Table 2.

You don't have to be an expert in every statistical test to keep an eye out for these 3 items. It's a good rule of thumb if you are a newbie to stats.

This paragraph is the first time the authors explicitly state that they are operationalizing learners' academic ability as GPA. In the introduction, they discuss that measures like GPA and SAT have been used previously but they did not state what they planned to use until now. This information should really be included in the methods section. Additionally, the authors have not told us what GPA ranges the high/mid/low groups have.

I'd like to know more about the score cut-offs for each of these groupings.

Note that these are tested separately even though they are included in the same research question.

I would have liked if more of this information was in the methods/data analysis section above. It's personal preference - I feel that results should be focused on the actual results, not describing the way you analyzed your statistics.

As Dr. Watkins mentioned earlier, remember to pay attention to statistical significance (p-value) but also to the practical significance based on the number itself. There are no set rules or guidelines for this so take the time to think through what the number actually means - this is key to be able to critically analyze someone's research/results.

Remember, each construct was measured using 5-point Likert scale.

I like to look at the table before and after I read about the results. It helps me stay focused on what's most important.

ANOVA tells us if at least two of the groups we are testing are statistically significantly different; however, it does not tell us which groups. Post hoc tests are a way to get this information. Scheffe is one of the more conservative post hoc tests and is commonly used.

Remember this when looking at the mean for each construct in the results section later in the paper.

It's always good to test a survey instrument before it's given. The wording of questions may make sense to you but not truly be measuring the concept that you hope to be (validity). Five seems small for a test but I don't have a lot of experience with piloting surveys just yet.

Response rate should always be reported. Simple but important.

I don't love that this study is aiming to compare engagement levels when two different professors are involved. The personality and/or instructional practices of each teacher have potential to be powerful variables that may influence engagement above and beyond if a course is designed as flipped learning or not.

Descriptive stats are easy to gloss over but they can offer valuable insight. I like to come back and read them again after I've read the results as they may offer inspiration for further study.

This list has all the key constructs for the study. I'm very visual, so I like to do a quick sketch of how these are proposed to be related to help me stay focused on the research question(s) at hand. (There happens to be one, Figure 1, below for this article).

Always take note of the sample size and the population. Remember, the results of this study can only be applied to the same population as tested in the study.

Again, talking about both explaining and predicting. Keep these goals separate as you read critically.

For more information: https://www.stat.berkeley.edu/~aldous/157/Papers/shmueli.pdf

Pay attention to words like "and" or "or" as these give you a clue for how they will be tested. Since this research questions uses "or" I'm assuming they will test the impact of epistemological beliefs on engagement separately from the impact of academic capability on engagement. Also, remember that there are many pieces to their definition of engagement (pre-class/in-class, behavioral/cognitive/affective). Since it's not specified it seems they are talking about the collective of those items.

I would have liked some citations for this sentence. How did they come to this conclusion?

I would have liked maybe a sentence or two more on each of these constructs. Just personal preference.

The authors still haven't told me how they plan to measure engagement. That's ok for this point in the paper but keep in mind that this is a really important piece that we need to know.

FL (design & implementation) -> learner engagement -> outcomes.

Don't be afraid to jot down shorthand of the variable trail being described! It may help you later to put more pieces together.

This sentence seems irrelevant to the study and the case being made. I often write down the population, variables, and research questions as they are revealed in a paper. This helps me to focus on relevant statements and block out ones that may not be as crucial to the purpose.

These next paragraphs look to describe key constructs. A well-organized article is much easier to read and comprehend quickly than one where the authors have you, the reader, hunt for key information. (Keep this in mind if/when you write your own.) Up to this point, I felt the descriptions of their key concepts were general - which is fine - so I have hope to get the answers I'm looking for in these next paragraphs.

This sentence gives the reader the reason for the study. It sets the context for why this study is needed based on the existing literature landscape. While I don't feel they made the case for "considerable findings" in what they have written thus far, it is helpful to see where they believe they fit in the growing research area of flipped learning.

While the authors discussed each of these "types" above, I was looking for a more clear discussion of how they conceptualize each of these concepts. Academic articles can be long and it's easy to lose track of thoughts you had early on, so figure out a system of marking or recording questions that works for you. I like to highlight in a unique color that reminds me to revisit an idea after I've read the entire article. For example, perhaps they do a better job describing these "types" later in the paper.

Play around with different techniques of reading academic literature to find what works for you. I like to read the title and abstract, then scan the rest of the paper for tables and images. This helps me to have a big picture idea of the goals before I begin the a deeper dive into the paper itself.

The title suggests this is a research study that looks to both "explain" and to "predict". Keep in mind that these two concepts are not identical. It's worth keeping an eye on how these were thought about in the research design.

Personally, I like that they gave some key statistical markers (test statistic, p value, and effect size) for each relationship - even when they weren't statistically significant. It allows the reader to understand the results a little better.

Note that anxiety is the only variable to decrease in mean from pre- to post-test. A decrease in anxiety is a good thing here - whereas an increase is good in the others.

The two-way ANOVA here (or factorial ANOVA) has 2 IVs (categorical) and 1 DV (continuous). Here the 2 IVs are time (pre or post) and type of teacher (language/humanities or math/science).

A one-way ANOVA has 1 independent variable (categorical) and 1 dependent variable (continuous). Here the 1 IV is time (pre or post) and the DV is the score.

Mediate and moderate are often used to describe relationships between variables. Learn the difference here: https://www.youtube.com/watch?v=WZr1jlKi_s0

I'd look up this source (Bitner & Bitner) as this sentence contains the major variables of the study and the source comes up a lot in the paper.

When reading a new article, always take a quick look at the journal name. You may start to notice a pattern of the journals that appear in the articles of interest to you, allowing you to sign up for updates from those specific journals.