Learning Outcomes Leaders 003 | Amanda Hagman

Welcome to Learning Outcomes Leaders.

In this episode, we're joined by Amanda Hagman, Adjunct Professor of Data Analytics and Information Systems at Utah State University.

We discuss how she's used predictive analytics to better support student retention, including one fascinating example, which involves a free drink at a sports game.

Let's get into the episode.

Transcript

Speaker: 0:02

Hello and welcome to Learning Outcomes Leaders, brought to you by Genio. I'm James and I'm Scott and on chatting to higher education talk about how they've been We'll discuss some of their projects and explore the most helped to improve and elevate In this episode, we'll be talking to Amanda Hagman, adjunct professor of data analytics and information systems at Utah State University. We'll explore how she's used support student retention, example, which involves a free Want to find out more. Let's get into the episode.

Speaker 4: 0:45

Awesome. Well, yeah, let's let's kick it Amanda, thank you so much. Um, for, for joining today. Really, really appreciate you Um, I guess Who are you? Uh, what's your sort of What's your origin story within Um, and maybe give us a real that you had, um, in education. What got you, what got you into

Speaker 3: 1:07

Uh, so my name is Doctor Amanda I wrapped up a PhD in And you might say, well, how is into higher ed? But it higher ed actually became my love and passion during my PhD. So one of the great things that I was doing is I was looking at these rigorous statistical models that weren't being applied at the university at all. And the thing that I loved more behavior across time. And at the same time, I was the university opened up to look Student persistence is a little retention when you're looking year over year. It's defined very specifically, and it also takes longer to get because you have to wait a full year to look at those retention numbers. So we took a unique approach by looking at semester to semester persistence. And we didn't only stick to the wanted to see how likely each one semester to another. And that wasn't just a model of, at the university? But it was really mission driven persist towards graduation? Because we know when people graduation, especially in the of debt without a credential, their peers who didn't go to because of that debt that's Um, and so I was able to jump stats background and this love Looking at that, we were able to find some really cool places where the university had leverage to help students persist that they didn't know existed before. And I was so surprised by all of um, work with collaboratively help students, um, move the in a better place, that they graduation and really leave with hoping to get at the beginning.

Speaker 2: 3:22

Awesome. I really thank you for. I think there's like a always, always confusion behind like persistence and retention, right? I think people think they go distinctly very different. Um, so it's really interesting focused on distinctly I have a follow up question Why did you choose to do that

Speaker 3: 3:51

Oh, you know, I actually have a really, uh, circuitous background. My undergraduate was in biology, I actually landed what I I was looking at And based on your diet, that wasn't a very eloquent way of saying it. But what we were looking at is we had mice models, and some of the mice mothers were fed really healthy diets, and some of them were given really bad, fat, heavy diets. Then after that, all of the mothers were exposed to cancer in a way that would give the mice babies an increased risk of cancer. And we were able to find that these mice moms who had really good, healthy diets, were able to protect their babies through their diet. And I love that this is Doctor She's brilliant. Um, but as I was doing that, I realized that the work that I was able to do in that space was very limited. And I wanted to stretch my own And when I really sat down to work I want to be doing, I stronger in the statistics space biological science space. And so I started looking around at other projects that existed, um, to get into it, to expand my education. And I came across this other Doctor Lockhart and she had the And it let us look at human behaviors, whether in education and business and mental health and substance abuse and how people change their behaviors across time. And I fell in love with that And so I moved from the statistics, but this very human people change across time. And that's really where I've my career, whether I've been in where I'm in learning and cross avenues across industries. Um, and I found that to be

Speaker 4: 6:11

So interesting that like, even, that was, you know, when you mothers then being able to their offspring when it comes to managed to find that that's such you've enjoyed across then and, and sectors as well. That's, that's so interesting that once you've found that you've been able to sort of keep that, keep that going throughout. And I guess in terms of looking at behaviors within higher education and, you know, sort of how, how you sort of came to, uh, to be at us, you in that role, what was your, I guess, experience of, of, of higher education and sort of persistence yourself with your, you know, your peers and things like that. Was it something that you were you know, we don't have a sense to have at the university or, certain things that you were you after when you start the role?

Speaker 3: 7:11

No. Um, one of the cool things is as we were looking at these models and we had partnered with, uh, a vendor is you could look at individuals for their persistence. And so I actually used myself a PhD student. And at one point, um, they used So if you're green, you're good If you're blue, there's a little drop down into the yellow and place to be. And at one point I pulled myself tool is work or works. And I had dropped from a green that I forgot to register for my my last semester, and it was in my and my, uh, assignment. But one of the. And so what it's looking at Is Amanda registering on time? Um, but there's other things that these, the model looked at and that we specifically did projects on that had to do with, you know, what are student behaviors like? And there was one really We, uh, Utah State University is located in the mountains of Utah. It is a beautiful campus so close to skiing during the summer. It's got beautiful dams that you Like it's a great place to be, but it is cold during the winter. And to walk around, one of the for one of the organizations are the computer labs And we found that if students went to computer labs instead of studying at home during the winter time, there was a significant lift in their persistence. And so computer labs, even that doesn't often get pulled in that, uh, finding your resources very different than people who during those cold winter months. And so then the, the, the, uh, computer labs were able to show how they're associated with persistence and they could encourage students to come and do their work collaboratively on campus to use the advanced softwares that were in the computer labs and just to even find the, the common spaces as a benefit, not just for getting an assignment done, but for how they're able to persist towards graduation. So that one was kind of a fun one of just finding the camaraderie on campus, using the resources and making sure that you are feeling like you're part of campus. And that's just kind of an that we did not anticipate a winter semester lift for those

Speaker 2: 9:59

That's that's super interesting. I was at a, I was at a Um, and they were, I was at a I can't remember who was talking, but they were, they were talking about a, um, a software which they use, which was basically like a, kind of like a social media for, um, the campus itself. Um, and they would monitor the things that were being spoken about. So one thing that kept coming up hiking kept coming up. So they created a, um, like a was all designated towards these people could come and go. Oh, I like hiking. Actually, we've spoken a lot about hiking and come to their to help build that like camaraderie around that same thing. And then they can, you know, I'm they, uh, bring in their coming hobbies and common interests all So it's really interesting to hear that you have that Like, it's kind of like a similar thing, right? It's like if, if they're around, you're noticing these things where they're all around the same place and that is affecting the persistence. What you mentioned a, um, the the, uh, the red light. Is that a, is that an early

Speaker 5: 11:15

Yes. That was the alert system.

Speaker 3: 11:16

It was an alert system advisors who are seeing, you persistence for students. They also had groups just at a, they could see, okay, this is yellows and oranges that exist And you could learn more about So depending on the audience that you were looking at, you had different levels of permission and different levels of granularity. Also, based on the types of So yes, the system had an alert, um, for you to work on those specific groups.

Speaker 4: 11:56

That's brilliant. I remember speaking with about, um, that, that sort of thing where they, they looked at even just the scheduling of classes sometimes. And they'd seen that I anecdotally nine a m on a Monday, students that were in those classes were less likely to graduate than if you put that then to say eleven a m on a Tuesday, because it was just a more time friendly situation, not just for the everyday student, but then actually the student athletes that might still be traveling back from a game, that sort of thing, and being able to have that level of granularity where on the face of it, student is signing up for a class, it's the same lecturer, it's the same class content that it's always been, but you can shift something so small by an hour or two hours, but the actual sort of in life experience impact that that can have on a student is can, can be really huge. It's, I think it's so good that can get so granular with this Um, I know you, you sort of the podcast started, you belonging and that being a real, for, um, for, for students. Um, I wonder if you could sort and some of the founding that side of things.

Speaker 3: 13:11

Um, so sense of belonging at a And because of some of the have, you could really see that strong, um, influence, years at university. And so freshmen who weren't able were far more likely to drop out a stronger sense of belonging. And We were working hand in hand with our athletics department and looking at the lift that students got from attending games. And one of the things that we and sophomore, that attending a in their persistence. And that's because they had this sense of belonging, a sense of community. And because we had this early a blanket, um, a blanket those who were a yellow and send a really specific invite game, you come to get a free that increased the number of Yellow, orange and red attending And that put them in a place where they could develop a sense of community and a sense of belonging within the organization. And for that small lift of a able to have a significant freshmen and sophomores that And even more significantly, they persisted from, um, we see the greatest drop during our summer months. So a freshman might go through And then we call it the summer And we just lose a bunch of those freshmen going into their sophomore year. But we saw a significant lift the summer melt returning, of belonging for the cost of a And it was a huge success at the

Speaker 2: 15:18

That's that's fascinating. How did so how did you track. How did you track the students How did you like the ones that How did you track that they were and were persistent from that.

Speaker 3: 15:30

We use a system where their into the game. So there's a student section at our football games, basketball games, a gymnastics meets, and if you scan, you're able to get in. And so because they have these who, um, who is attending. But the reason that we needed to organization is because they data, their student ID scans. And so by working together, we were able to break down a data silo to show that these types of scans coming into basketball games and football games and gymnastics meets those all have a sway on student persistence and data that exists at a larger university level. So working interdepartmentally became really important for these types of analyses that we were doing.

Speaker 4: 16:27

It's fantastic. I think it's such a specifically it comes to college sports. We have nothing like that in in And my sense of belonging really my group of friends and, and and the fact that I liked the, actually that sense of university and an organization isn't, isn't there as much. And for, for people that are sports jerseys behind me. Sport is a big part of my, um, life, but I think it's something that's so amazing that you can all of a sudden have a student that, you know, might not necessarily feel comfortable on campus. You know, it's a new new town, higher education can be quite intimidating and then all of a sudden be surrounded by sixty thousand like minded people once a week. All of a sudden you get that yeah, I'm a USC student or I'm a This is, this is unbelievable. I'm part of this, this sort of I think it's, it's so yeah, I a bit of a random tangent, but I think it's such a specifically fantastic thing that, that you guys do. And, and was this something that you were doing sort of as a one off? So it was get your first drink Or was this something that try and get students back

Speaker 3: 17:53

Yeah. So this because we looked have, um, counts of how many versus James or versus Scott. So we could look at that by like Like, is it better to be a one timer like Amanda? Um, and so we were able to do We did not do it in real time Instead, we looked at this semester to semester granularity. Um, and I know that there were largest lift was between those at least attended something. And so looking at those two groups, um, we saw the largest lift. And so we weren't really fanatic, but just helping them and knowing that that exists can use it. And I think, James, going back because, you know, in the US, we traditional students, but we non-traditional students where, complete their degree, they We're really trying to open them back in. Sure, they may be in their forties or their 50s, but we want to create the same experience. Not necessarily through basketball games, but help them to be persistent towards that end goal of having their bachelor's degree. And so finding those right group that they live in. It doesn't necessarily mean that they both started in twenty twenty five. So they need to be treated the But our non-traditional students amount of vigor. But for the lever points that whereas an eighteen year old may need to be turned to help them persisting towards graduation.

Speaker 4: 19:58

Yeah, I think I think that's I think specifically with generation students that are experience of, I'm going to go my dad went, that's where mom So that's where I'm gonna, you things like that. I wonder how sort of widespread sort of looked at. Another thing that's sort of America, that sort of, um, groups and things like that, might be, I might be wrong. Please do correct me if I'm again, that's something that fifth generation students are and have that sense of belonging Is that did you ever sort of go down that route, sort of looking at that membership to sororities and how that how that might have impacted things?

Speaker 6: 20:47

I believe we looked at something

Speaker 3: 20:49

It's not fresh on my mind. And Utah State doesn't have we sororities, but it's not quite know exists in other locations, But I do know that, you know, especially as they create a service, really do help lift Um, which can be huge. But again, I don't have a

Speaker 4: 21:21

Sure. Yeah, that makes sense. Thank you for that. Go ahead. Scott.

Speaker 2: 21:24

I, um, so you mentioned you was, um, there needed to be a, down to inter departmental work. Um, talk us through that. What were, what were like some of the challenges of, of like pulling this data together, getting this project up and running?

Speaker 3: 21:45

Oh, that is so it is huge. So I, at the time, I would call it really lowered, um, lowered by my peers. But I would always tell people a data wrangler, it's fifty percent wrangling people. And so you have to get the buy in of the individuals doing this. One of the mistakes that we made early on is we went to a group and we said, hey, we've got this really great tool and we're building out this department for student success. It was a student analytics And we came in and we had this really robust data on their pro program because their data was not siloed. And so it was something we could And we came to them, we said, And all of a sudden these walls And it wasn't something I was especially a PhD student who sudden there was a power things about your work that you And so we scrapped that model there were there was a ton of We decided to do the legwork this new tool. Can we run it for you so that having towards persistence? Instead of coming and saying, This is the impact you're having because we needed that upfront buy in. And we really took a very unique the tool had the power. We could do that. We had management and leadership We could have rammed this down But instead, we wanted this to be a grassroots growth effort where, yes, we were supporting the departments, but then we wanted those department leaders to then walk away and use that data to support their cause with their leaders. Um, and so we had to be very rolled this out because, uh, universities, people really They are experts, they're highly in there and just telling them numbers actually look like. Um, didn't have the impact that And so making sure that we were very much people centric and helping them solve the problems that they wanted to became a huge part of helping them show the success within their organization. The other piece that we would do, because the data analysis that we used was very quantitative, we had certain access to data that was easily quantifiable. But there's also all of these good student stories and experiences and qualitative data. And so we would also work with them and say, okay, this is what the quantitative story looks like. Do you have anything that you'd And they'd said, yes, there is Z. It really showcases the type I want to put that in there. And so as we were pulling we would open it up for does any of this surprise you? Is there anything you want to Is there anything that you don't And I think that's a really big professional, that the data only you actually care about. And some of those stories, even though they're not easily quantifiable, they are just as compelling and really strong examples of what you're trying to do at the organizational level. And so being just data driven shuts you off to some really important insights that you could have. So those are a few of the it out across the university. And we had fantastic success in getting people to adopt this new technology.

Speaker 2: 25:49

It's it that's, that's great to Like your people focused. I think data is scary for a lot It, it, it, it's so useful to understand where you could get better. But By saying that, you're also doing so well. And I think that is scary for a And having it all laid bare like Um, you need someone to come in approach and be like, look, what We're trying to help you. We're trying to make you better. And we need to understand this What, uh, what things have you the space do because of the data

Speaker 3: 26:41

So a huge success again, was So it was specifically winters. Like I said, it's cold. We want to get people up and feeling engaged with the organization even during the winter. So it was basketball that really at the games. I feel like that was just such switch that could be turned. Um, other huge pieces that, uh, was that they were able to look that they were actually spending were already green and blue. And the students who were yellow, orange, and red were only getting a fraction of their time. So they were able to strategically go in and say, you know what? We need different communication orange to proactively pull them that they can talk through this. One of the things that was is true about young students is that they are may be in a major, that they are finding out that they don't love, and that can make them question, maybe I don't fit at the university, but if they can have those early conversations with an advisor, they can find a place or a space that's more meaningful and aligned with what they want to do professionally. But when they're left to solve feel like misfits. And so the advising group were able to proactively reach out to students using, um, you know, different scripts that they that would ring true to these groups and invite them in for personal sessions. And we did see an increase in the number of people switching majors, but that's not a bad thing. When you switch majors, that that you're still exploring what So we were able to have a lot of success also with our advisors doing proactive outreach through this tool.

Speaker 4: 28:44

That proactive outreach, I think We were speaking with someone recently that had a very similar experience in terms of you're looking at your one to ones that your academic advisors are doing, and it's all the students with 4.0 GPAs, it's all the students that are doing really well. And just, you know, it's helping It's helping them. Same in the writing center. It's I don't think this is quite And the students that are really often don't necessarily know I, I remember as a student with, with, with a disability. I had no idea where the office I had absolutely no idea. And I didn't know that a writing what was available to me. And actually being able to have that sort of proactive thing of, hey, here's a pointed message to you in your experience that that you're going to feel that that student is going to come out of the woodwork wherever they are and go. That's that sort of support that I need, particularly those sort of non-traditional students that that don't have a parent or, you know, that has been to university and gone try this particular, um, department as well. So yeah, I, I really like that. Um, I guess moving on to sort of the next bit, I guess when you look at the project that you sort of did as a whole, I know you talked about some of the learnings there of, um, you know, don't come to people with the data straight away and go, here's what I found about what you've done and what you should do differently. But have you got any other sort of, um, learning and advice if, you know, if someone listening today is thinking, you know, data is what we need to do, we need to jump straight in and really sort of find these moments as someone starts that journey to, uh, towards doing something like, like you've done what would that, what would that advice be?

Speaker 3: 30:28

Yeah, I definitely remembering of what you actually care about. And so stepping back and trying to imagine what the people are actually doing. So we looked at, like I said, swipes mean very specific things It means a sense of belonging at It means something that they're progressing, that they're utilizing resources, that they're finding the resources on campus. Um, and so trying to understand level can be really huge. So that you understand what people are actually trying to accomplish when they're doing that behavior. And so making sure that you put that human lens on it, I know that we're moving in a strong direction with AI and AI tools, but doing the exact same thing and learning from some of the fumbles that the data industry has had, especially with the, uh, the data driven, data driven has never been a term that I've loved. I always like data informed or data backed or data supported, because data driven means that you're letting the data make the decision. And the data is only a reflection of the things that we care about. And so you still have to use your human judgment to make decisions. So even as we move ahead, away from these basic machine learning models like I was using, they're not basic, they're advanced machine learning models. But compared to what AI is we are doing a very similar humans behind it, and not just decisions, but help it help us that they're giving. So if anyone's picking up any of these automated techniques, whether through machine learning or through AI, making sure to have that human touch and really understand what they're trying to accomplish because you're impacting humans, and these humans are taking on huge investments in their future at your university. And so offloading decisions can you're offloading it for your heads, for your employees, or You have to be very thoughtful. And so I would encourage you to that's data driven, AI driven to it's having so that you can abilities and helping your or students, achieve the

Speaker 2: 33:18

I yeah, data driven gets me a I think the data driven for me feels like exactly what you just said. It's, it's, we're going to do should and we're going to doing this is. Yeah. Um, we're just going to do it should do this. Whether that means, you know, five people which really rely on like a human centered support are no longer going to get that support because the data says the rest of the students don't need it. So these five students aren't It's. Things like that are, you know, it's just tunnel vision effectively. Yes. Tunnel vision just for the data. I have really enjoyed this I really want to learn from you What are your hopes and dreams on higher education?

Speaker 3: 34:14

Hopes and dreams of the impact of this type of research is that people can be thoughtful in what is most effective for student success. So I mentioned several analyses There were others that didn't And as we are moving into this going to change immensely. And it doesn't mean that the research that's been done is less valuable. It's actually more valuable that thread from, this is what Is it having an impact on where And with the the workforce work being so unstable, with such a a static infrastructure are going to need to be more more adaptive than ever. That means you need to be able inputs, what we're doing as students in their outcomes. And a lot of what is being done to be questioned, and it's going And so having an eye towards evaluation towards measurement can be really important, especially in these times of transition. So this is what I'm hoping to I know that Utah State is conversations of what our What experience are students in the AI era? What are they going to need to learn differently or in different ways so that they can really springboard into employment? And without those conversations you're really in jeopardy of not putting the organization in a, So those are a few things that I'm hoping that we see in the future.

Speaker 4: 36:30

Yeah, I love that. I think, yeah, that, that forward thinking approach that we're in a huge period of transition. We need to make sure that we learn from what we've done in the past, take that forward, but be agile. I think that's it's, it's a lesson, I think for, for all of changing so quickly, um, I read is probably going to be a bigger Revolution when it comes to the work and live over the next, Amanda, thank you so much for This has been really, really Um, I know I've certainly I know Scott's enjoyed it as I can see him nodding along. Um, thank you so much for being We really enjoyed having you.

Speaker 3: 37:15

Thank you so much, James and This has been an incredible Thank you for inviting me.

Speaker 4: 37:22

Absolutely.

Speaker 2: 37:23

Bye bye.

Speaker 3: 37:24

Bye bye. Bye.

Speaker: 37:28

And that's a wrap. Thank you so much for joining us We hope you found the conversation valuable and that it gave you some ideas for how you can elevate your own student experience. Don't forget to hit that get your podcasts so you never Until next time, have an amazing