Learning Outcomes Leaders 008 | Heather Chapman

Welcome to Learning Outcomes Leaders.

In this episode, we're joined by Heather Chapman, Senior Director of Data and Analytics at Weber State University, to discuss why we need to look beyond academic grades and start using co-curricular data to measure true student success.

We explore how predictive analytics can unlock meaningful insights into the campus experience, and help institutions support students more effectively.

Let's get into the episode.

Transcript

Speaker: 0:02

Hello and welcome to Learning Outcomes Leaders, brought to you by Genio I'm James and I'm Scott. And on this podcast, you'll hear professionals who can talk the there and done it. We'll discuss some of their projects, and explore the most helped to improve and elevate In this episode, we're joined by at Weber State University, to beyond academic grades and start true student success. We explore analytics can unlock meaningful insights into the campus experience, and help support students more effectively. Want to find out more. Let's get into the episode.

Speaker 1: 0:50

hello and welcome everyone to Um, we're joined here today by Heather Chapman, who is the Senior Director of Data and at Weber State University. Heather, thank you so much for joining us it'd be great if you could just us a bit about your origin brought you into higher in the first place?

Speaker 3: 1:13

Absolutely. Yeah. Um, I'm actually in psychology. Um, I went through my undergrad, didn't really know what I wanted to do after that, um, got into grad school and figured out that I was really pretty good at statistics and I liked it and it was good. And so I focused on that as I came out kind of before, like the like data analytics boom. And so there were kind of Um, I got into an institutional institution of higher education education, I bleed education, So that's really where I was Um, and that's kind of what got of this chance job in as an Um, getting into that, it became people didn't really care what like the, all the fancy analyses Um, what really mattered was whether I could tell the good story. And so that really is where my career trajectory took off from there. From that point, I've just been focused on trying to take these really complicated analyses for people, no matter what it is about and, and, and, and turn those into a story that that resonates with people that don't have a statistics background or don't have a data background or don't do that as their day to day job. So that is what has got gotten kind of been my focus.

Speaker 2: 2:55

Really interesting. Thanks for that, Heather. I'm just interested because you mentioned your psychology degree and I'm by no, no stretch, you know, a doctor in psychology, but I also did a psychology degree and I found the statistics element really daunting. So that was just something that, similar experiences, although I found stats daunting. So was that something that you liking to and like focus on Or was it kind of like an after

Speaker 3: 3:26

Yeah. No, not as an undergrad Um, as an undergrad, I was like didn't know what you know. In any field, you have like or social psych or whatever. And I really didn't know what I And so I got into this program evaluation focused program. And so part of that was research So I learned a lot of that, and then statistics was like So it wasn't until like year two of that that I was like, oh, this is why I didn't know what I wanted to be because I'm actually really good at the numbers. And so it just kind of came and actually, I think it was when my professors that was like, you know, will you come be this, you know, this, this grad assistant role? And I was like, that's in stats. And he was like, well, yeah, And I was, and so it was really kind of this again, like I've had a lot of these just like fortuitous things that I've fallen into and said yes to these things that have really shaped me. So no, by no means did I go into even third stats class and go, be when I grow up. Um, it, I grew into it, but, it all makes sense looking backwards. But looking forward, I was like, So yeah.

Speaker 2: 4:37

Yeah. That's fantastic. I mean, if you're kind of into the numbers side of it, I also found the research side interesting. Um, but it's the numbers. It can be so daunting. But yeah, that makes perfect

Speaker 1: 4:48

I, um, I always found maths and Um, as a student, I, I stopped, uh, maths at high school and it was something that never, never sort of piqued my interest much until sort of the last few years. And when you said about using think it's only really in the I've actually realized the power start building that picture. Um, for people specifically, you know, like me, that doesn't necessarily have the overall grasp of how we got to those statistics. But being able to say here is, background methodology and the And this is, this is our findings and potential recommendations. How how have you found that sort of telling the story side of things? Because I'd imagine that you have two types of people, maybe one being like, great, all these statistics. I want to know exactly what analyses you use, what were your methodologies? And then the other that went It's all numbers. How do you find that?

Speaker 3: 5:48

Yeah, you're absolutely right. I think there's also a third person that is the person that really likes numbers, but doesn't necessarily always use them in the best way possible, right? That digs in. So, um, I kind of always take like a lowest common denominator approach. The good thing about stats, this long, but is it's logic. It's a lot more logic than it is So this is actually something I, I teach as well and I teach statistics. And, um, one of the things like the thing I say on the first day is like, how many of you are afraid because you think this is a math class and they all raise their hand and I say, well, like, lucky for you, it's not, it's a, it's a logic class really? Um, because for most people, do, it doesn't matter. It matters that we follow the So for most people, if you have background, like doing work in can get around like the You can trust that somebody like me is going to be like, well, I think we should use X, Y, or Z method. Um, and then you can just focus And that's the part that is the hardest in some ways, but also I think where, where like the casual user can put their effort. Um, for me, the critical piece is to kind of hide all those details. You're right that you will have but it's like three, right? Like on my campus of a couple probably like five that want to And they're probably also stat But, um, so you've got those know everything about the model decided which model to use. And so you need to have that people can check you, right? It's like a reference. Um, but beyond that, it's okay. Nobody wants me to talk about like the root mean square error approximation. Like nobody wants me to talk What they want me to do is say, okay, what should I take from this? So what we've done and really those predictive models and everyday dashboard so that we specs document or something like and look them up and check us But then people that are accustomed to playing in our dashboards, like, or just our everyday, you know, demographic dashboards or trend dashboards or whatever, um, they can go with to this dashboard and say, okay, well, let's see what the predicted, you know, what happens if we combine these different metrics or these different demographics. So I can see what is the effect if I have these that are my struggling learners, um, what happens if I give them a job on campus? Um, and people can see, oh, the Um, and so we've, this is, it's to do, um, I think pretty good complex stuff, having it built letting, uh, basic, I mean, it's outside looking in, it looks everybody's used to. So that's, that's the approach Um, and we've had limited So like there's, there's pluses Um, and we still have a long way just put it out there and then insights, like you still have to we're still perfecting things. Let's just say that.

Speaker 2: 9:26

Yeah, it's fascinating because, you know, it's got the potential to be really impactful though, doesn't it? You know, if it guides how you know, support students. So I'm quite interested then what are the kind of metrics that you're using or like how or what kind of, um, kind of variables are you measuring to predict student outcomes and things?

Speaker 3: 9:44

Yeah, absolutely. So we really have four models Um, and then there's, there's mean, in higher education, it's So we're open enrollment. So we get a lot of over So we will get ten thousand applications for fall and maybe like three thousand five hundred or four thousand of those will actually come. So it's, we have one for Um, we have one for, uh, that first year. And then we have one for Um, we've also gone into engagement to look at what is the impact of engagement on students and how protective is that? So each of those models has different metrics that we've looked at. We look at all of the basic ones Like a, like a sex or ethnicity or, um, age, you know, I think would, would normally think of, um, major or they're declared things like that. We've also looked at things like So for instance, in some of our the, the, the impact of the we are in or of the nation. Um, and, and look to see if those have an impact because we find that like our enrollments go up when the economy like starts struggling. And so we have tried to model We model things like their Um, you know where they're coming from so we can try and see if are there pockets that we should target. We look at whether they work on campus, whether they seek out advising um, some like what is their preparedness when they come that first day and walk in the door? Are they, you know, do they need like some of our remedial math or developmental math or English courses, some of those sorts of things. So it's a long list. We probably use, I don't even the model, somewhere between in the model and then just kind want to use.

Speaker 1: 12:03

It's really interesting because I think there's, there's been a real sort of boom of, of this statistical analysis. I think you mentioned this You know, you came into the of, um, this, this boom in this data is. um, on a, on a couple of done, um, we sort of touched on We, we met with Amanda Hagman who works at Utah State University. Um, one of the things that they student retention through the student had attended one sports sport or anything like that. Um, but because that sense of belonging was there and you didn't feel like a number anymore. Um, that was huge. So one of their things that they can get a free ticket and a free to really sort of dial in on, on And then we spoke with David and Arts, Oklahoma, um, tiny, students in total, very sort of And one of the things that they actually look at what particular gone to and then actually start say, well, so and so high struggle with maths more than them in for classes like that. So some really sort of different What have been some of the most, that you've had so far that you big impact on? Yeah.

Speaker 3: 13:39

This I can get really geeky it down, but.

Speaker 1: 13:43

Let's get geeky, I like it.

Speaker 3: 13:45

The one I'll stick with that I'll start with is I'll try and be brief. But, um, we have this narrative on our campus because I mentioned we are open enrollment and so we take anyone, you just send us an application, we will take you, which means we get everyone. So we get, you know, students could go anywhere and for some You know, they want to stay And then we get people that are And then we get those people that may have struggled through tried like a community college Um, and those folks will often Um, I mentioned this before we in math or developmentally Um, we had this narrative on developmentally placed math And it like existed like before I got here existed, like while I was here. And then we ran these analyses is not what the problem is. It's those students that are English and math. And so that was really a game We were able to kind of take that like colloquial, like common sense knowledge that people run with a lot, that it was these developmentally placed math kids or students that, that, um, that were, that were having, that were really struggling, you know, because we, we would always have these narratives like, oh, some of these students are taking eight times to get through math and like, yeah, you have those outliers. But you know, this wasn't the So so we were able to show that actually, it's, it's, it's math and English and start a whole program. So we call it Wildcat scholars. And we bring in these kids that are these students that are placed, um, in, uh, developmental math and English and put them into, if they want, it's opt in, but into a cohort that follows them for the entire first year. And we've had really good You know, we took our from like twenty or twenty five percent, um, in these students. So, um, that's probably the one I think that we've been most excited about, but we've also used it for a couple of other things. So for instance, we were able to show that, um, meeting with an advisor. So because we're open weird ways that that gets like, campus, I guess. And one of those was like, well, therefore we can't force people Like we were really like, like of any of our students. And so we were able to use the well, those students that attend are retaining at this ridiculous or something like that. So, um, we were able to then get, uh, funding for additional advisors so that we could make our both new student orientation and mandatory first year advising. Um, we were able to launch both of those programs, which was for us, something that we had never done before. Um, yeah, I could probably go just in case.

Speaker 1: 16:55

How does, how does that, um, So you said that that's now mandatory for first year students. Is that how, how often do advisors and things like that? How does that work?

Speaker 3: 17:06

They have to meet with them as they're just as they're coming in. So we yeah, haven't moved it need like a, you know, the second semester advising. But what we're hoping for and was the first semester that we Um, and so we haven't really seen the, there hasn't been quite enough time for us to be able to see all of the benefits of it. So we haven't done the analyses yet, although we should be able to now. Um, so I don't really know, the positive, but, but we didn't, we Um, so these students basically their new student orientation. And then the last part of that is that they have to meet with an advisor so that we can just get them set up for the right classes. Because what we were finding with our students is a lot of them would take like, I don't know, like anatomy and their developmental math. And they're like, I don't know, like whatever other like history, history is a really hard intro. History is really hard on our And then they would find And so we we have used that. We do know that like our we've been able to kind of help their first year and kind of Um, again, we'll, we'll see if really how it's been

Speaker 1: 18:31

I really like that as an idea. I think one of the challenges spoke about this on a, on a to quite a lot of people about great, but one of the challenges the students that go to the appointments tend to be the ones And so it's great to have that But the biggest impact that we maybe aren't as aware of the that, that support level. I know even for myself at university, I, I wasn't fully aware as a student of all the different things that that were available to me, I would have done so much better at college if I was aware of of all the different things that, that were there. I walked past an office and all and read it. Yes. How?

Speaker 3: 19:22

It's hard. I mean, it's a it's a challenge. Like a most of our campuses run And so when you think about moving to a new home or an that, and you have to turn on and like, get the utilities on all of that. It's like, it's kind of And a campus is really, really Like there's just so many And we don't have the same understanding as like a, as, as people or as like coming in, right? Like you don't know what you So yeah, I hear you.

Speaker 2: 19:55

Mhm. Fantastic. Well, I'll just that it kind of follows on a there a kind of student outcomes of, and why was it that you

Speaker 3: 20:09

Um, let me think how to answer this one, because I probably could answer it in a couple of ways. I think, um, I'm most proud of build it out. I don't know if that's what that this, and then I can answer rather, but, um, when I started, It was me and this was my It wasn't what I was hired to So I believed in it. And so I would just push these little projects here and there and be like, look, did you know this? And, and then I got some people on board. And we did a couple of these like large, like enterprise type analyses and got more people interested. And then, um, honestly, Covid hit and I, a local company reached out and was like, hey, we'll give you this product for free during Covid so you can try it out. And because we're trying to help good marketing for, for them. Um, and I, so I did that and, and built, and I for, for that, I was actually able to take this to like a machine learning model, not just like a traditional predictive analytics model. So it was a more robust model, Um, and so I did that and I was able to show like such greater outcomes. Um, and like just much stronger build up funding then to buy from there, it's just like I we've added a full time FTE which we, you know, is like kind Um, we've moved to like a Python infrastructure into our, um, our results in there, easily auto populate and auto update So like there's all sorts of infrastructure that I'm really, started as such a like hobby. Um, and now we're on the brink. We're not there yet, but we're I can feel us like we're pushing the hill, right? Or the top of the hill so that able to provide people with some to kind of seek out the students and see some really like, um, than just guessing. Uh, so like we're right at the

Speaker 1: 22:38

It's so exciting. I think that being able to turn that you're interested in to all sudden it's took a long time and But to get to a point where now basically live running through dashboards and, and, and allow self-serve sort of thing for for in is so powerful. We've, we've sort of been on a similar journey ourselves with with data. And it's amazing what you can available to you. So, you know, whether it's how sort of particular state or large scale, being able to sort constantly iterate rather than, thing, you're, you're doing institution going, oh, two Um, why you can see those early, things like that. I wonder what were sort of some of the main challenges or, or blockers you've had to something like that because it's a, it's a total mind shift for, for, for the institution, I guess, or a mindset shift. What, what sort of things have

Speaker 3: 23:52

Yeah, that's a great question. Um, and one that I think we will think what the, I think that the when it comes to data is like an So we've been like, we're We've been very successful at like, we can answer the most And, and not even they don't like, like our, our faculty and staff don't really need us for that. They, they have dashboards out They can answer so many Um, and that brings more questions and more questions and more questions. So eventually you get this kind of glut of dashboards where you've got these, there's a ton of dashboards and they can do so many things. And so I would say that is now is even when we have these where people can, can look and here are my murky middles, kids And this is what, um, you know, focusing on with them that gets um, particularly speaking about will human nature, I think is to Um, because people can talk about it. And so when I provide them with something that they can't validate in a trend, then it, that's a challenge because they'll say, well, but the data shows this. We've grown in this and this, actually bad or whatever. Um, and so there's this cross play between just our like descriptive dashboards and these predictive results and trying to figure out how to talk to people in a manner that helps them to use the two together, um, and not see them because we have to change the narrative in their head, right? When you're looking at X, Y, or Z dashboard that we've, we've taught them and they've learned, um, okay, this is how I use trends. This is how I use, you know, these data over here, but we're still in the process of teaching them. How do I use this predicted And so it's there. It's a little bit of a shift in And I think that that is a Um, because it's like this new schema or this new frame of reference that they need to just learn. And so it is very, it's, it's

Speaker 2: 26:17

Um, yeah, that's really just, um, kind of what the stat Sorry, is it? It's also like uncertainty then worked in a certain way or trends in a certain way. And, you know, it's, you know, So I can imagine that is the, the rest of the challenge almost once you've actually got that statistical statistics. Um, but yeah, that's Yeah.

Speaker 3: 26:41

And I'll just, sorry, both of yeah, I do stats. Both of you said, yeah, like I I wasn't thrilled about stat. Um, and that's also kind of what So they go into it thinking, I And so they default back to this thing that they've already done, even though they might be able to get a little bit more information or like more precise information. So that's what we're dealing Like most people have the Like, this is hard. I don't know how to do it. I'm not good enough. Right? It's like that imposter I'm not good enough to deal with And so it's, it's addressing it be as accessible as possible.

Speaker 4: 27:28

Yeah. It's a shame though.

Speaker 2: 27:29

Sorry, James, not to interrupt. So it's a massive shame because doing the, um, so I did research I was a research assistant, um, And that was something that I on, but I was like, I need to when it comes to just lot of it is, of course is doing need lots of part with it. So I mean, it's how do you how Like, how do you encourage them they can do it? I know it's so complex. It's such a, you know, it's not But yeah, it's it's just a

Speaker 3: 28:09

Yeah. And, and data is like Like, you know, we had a gold rush and now we're in a data rush. And the companies that can leverage that are the ones that are succeeding. And in at least in the US, like hard for all of us out here. And for some people, it's We're competing for for for, you know, dollars like all So I think higher ed is in terms of how many like that are out there. And I'm telling you that like the, the institutions that are able to harness this and embrace it and accept it and embrace the data and use the data to drive who they should be going after and what they should be doing for those students to get them to succeed. Like those are the institutions that are going to be like flush, right? They're going to be have such is just going to grow. So like, that's why it's For me, again, I can get so like, that's like, I could not The, the, the institutions that to be the ones that, that are ride this contraction in the US. So from my point of view, of

Speaker 1: 29:31

We, we talk about this a lot, the, you know, the, the that's coming very soon. Um, and we're already seeing, of small private liberal arts merging seemingly monthly, And so we've we've had this huge institutions and numbers of But then as the as the demographics of students in terms of numbers are changing, we're seeing really big changes in terms of who's going to, to university. And we've, we've started calling new majority, because when we is so many first generation disabilities, for example, coming back into education. And so not only is the numbers changing, but it's actually that that entire makeup of the population. So yeah, I completely agree with you in terms of the institutions that are actually really looking at that data and, and really being flexible with what they do next rather than going, well, this worked from twenty ten to twenty twenty. So we're going to keep doing And then Covid hit and things are different, but we don't know why. But that worked before. Um, it's, it's, it's a really You've got all these years of knowledge and then the world seems to have changed it immediately. What what have you seen so far? The institution has had to do or the industry has had to do in the face of the enrollment cliff and things, I guess post Covid where it's really sort of changed.

Speaker 3: 31:07

Yeah. This is this is a great, Um. Yeah. So I think for us, a lot of, uh, really kind of doubled down on So, um, most of our people are So I would say like we're eighty So we get some outsiders and I but, um, so we've really tried with like our, um, community We've also worked in the state to do things like direct admissions, which is not new, but it's kind of new for us and some things like that to try and remove barriers for students, um, and try and go after those because I think you're absolutely right. The demographic is changing. Um, I don't know when the dust all settles what it will look like, but I think that we are seeing, and I'm not like unique in saying this, but we're, we are also seeing it that like the traditional like nineteen eighties or whatever, um, students like the white middle class student coming to school, those are the students I think, that are more skeptical these days. And so for us, we're seeing So, um, our next demographic here where I am is our Hispanic, um, group and we're seeing gains there. Um, we're seeing gains in our first gen, we're seeing gains in in, you know, some of these different areas that we would have traditionally had to really like strive to recruit, if that makes sense. Um, which I think is wonderful. Um, I think we still have a lot now, like, because things are shifting, we have to figure out how to get to those students that have become like, I don't know if like skeptical or disenfranchised or like, I'm not sure what it is, but we've, we've, I don't know that we have an answer for that, but we're going to have to start figuring that out. Um, I'll also just point out like another area is those that have some college, but no degree. Um, and I think that's also a buzzword currently out there, but like, I'll just throw out some numbers. Like we, again, we're open So, you know, I was looking the twenty thousand students over have stopped out. At some point they came for a semester or longer and they've stopped out and didn't go anywhere else. Right. We, we can we can see that through the National Student Clearinghouse. They didn't go anywhere. So they came to college, they happened and they thought, this And so that is also, I think, a really big opportunity for us to go out there and figure out like, what is going on with these folks. Um, what is it that has Is it that they came and that like life happened and, and something, so they had to go a different direction? Or is it that they came and they just those like mental health in and I'm thinking I'm not good And so the stress of that is, is folks in their twenties, right? Um, anyway, I think that's a could go after. And many of them started and and now they're maybe older. So then yes, we get that like you talked about. So I think all of that is in play now and we as institutions, at least where I am, we're going to have to be creative because the traditional. I feel like we give lip service to this, where we say, we know we're changing. And then when we look at how we're organized as institutions, we're still open nine to five and we're, you know, not open on the weekends. And we, uh, you know, we're from the hours of nine to two. And like, there are all these to address the issues that these these students that have left Like we, we've got to have this like reckoning of like, we are giving lip service to this idea that we're no longer the same institutions. We've got to make our services in the same direction.

Speaker 1: 35:31

Yeah, I had some really good I was at Student Success US couple of months ago. And the, stop out students were quite a lot where we were two types of students. It was the new students fresh university and have missed a lot of high school, sorry. Um, through Covid. And so their critical thinking actually lower than, um, And it gets quite daunting. And they go, I don't think I can they need and stop out. You've then also got those things like the cost of living going, actually, I can't afford I need to be at work for, for, Um, and, and, and that sort of that sort of side of things as well. The other side of it that we, quite a lot is, is the changing engaging in classrooms. And you sort of touched on that And, you know, we, operate nine Some people are going down sort students can go. Um, log on to a classroom from Is that something that you're Are you looking at it in, in I could go down a whole avenue And, and things like that. I guess Hyflex first is that, is that something that is a priority? Are you looking at other ways

Speaker 3: 37:04

Yeah. Um, we have looked into Um, we've looked at things like, okay, so let's maybe let's offer like, I don't know, like a two week class, like a really intensive two week class or let's offer only Saturdays or let's offer, uh, I don't know, like a bunch of alternate schedules. So we've looked at a lot of Um, we also have like our both asynchronous and synchronous online classes. Of course, the, this is a really tough one that I think, I think we're still in the throes of because you really are fighting two battles. Like it's a it's a war on two a, um, this, you've got the So I'll just use this statistics like, like if you look at our, our institution, we're primarily like, uh, just like regular brick and mortar, we're not online. Um, but if you look at our, our percent of our students are least one at some point, right? So like most of our students are that are taking online and face Then you've got kind of this traditional idea that online is somehow inferior, or we don't want to be just an online institution. Um, and so you're kind of Um, and, and the faculty who just always been the case. Like you have like, not to be all like ageist or anything, but like you have older faculty who came up in a certain way and are used to learning in a certain way. And then you have younger children coming up saying, And so you have this. It's this like friction, right? And so there are those sorts of that keep us from being as these hybrid type formats. You'll have your innovative is what the students, I'm going I'm going to do it to the best And then you have faculty who are like, this is the way we teach X, Y, or Z, um, like program. This is the way it is done. You're not going to go out in the field and, and see it any different. This is how it is done. And so there's this constant battle, and I don't know that it's any different than what it might have been like prior to now. I think it's much from my point of view, just given a lot of the challenges and things that have like, you know, Covid and like economies and like things like that, like cliffs, all of these things. Like given all those things, I. From my point of view, I think it's coming to a head in a different way than it has in the past. Um, but I think we're still kind think we're thrashing at it, if

Speaker 4: 39:49

Um.

Speaker 2: 39:50

So do you think just out of large, you know, from this or model is largely to do with like Or where do you think that kind Or like, why people don't want from your opinion?

Speaker 3: 40:07

Yeah, I think there's this cannot be, um, as engaging and experience and the same class taught with an expert in Right. Um, and so I think it's partly expert in this, but I just think of like, online is different. What you do is different. The, the, the, the, you know, with people, which we know when you got to go get jobs and you So that's like, we have to have So how do we balance those sorts of things like, um, like research methods. I teach research methods. How do I, how do I get them engaged in a research project with a group in an online environment? And I'm sure that there are people out there that have done this. And so it's not, it's more of a, Like, I think that's what we run And some people have like the and some people don't, right? Because some of us have passion just working a job. So like, I think it's partly some of those things that come into play.

Speaker 2: 41:20

Yeah. No.

Speaker 4: 41:21

Of course.

Speaker 2: 41:22

Sorry, James.

Speaker 1: 41:23

Yeah, absolutely. It's it's a big challenge. And we, we sort of have this as So we're, a company of about one And I would say maybe sixty five around um, Leeds, which is the Everybody else is remote. That's right. The way across the country, plus And so working in this sort of hybrid environment where some people are going to conferences, a lot of us will go into the office once a week, but then people like Olivia live in London. And so we'll, we'll see each other once every month to three months. Um, and so it can be really belonging, have that sense of And we've managed to get around it or start to tackle things like that by making sure that people are having those moments together. You know, each team has a budget activity, a team building thing just to keep that, that sense But I imagine it's a lot more space where you're trying to get things, but they're also a Rather than wearing an employee go and do this because it's do it, and it is great versus a these other things outside of may not necessarily have that like us that is paid to be Um, so yeah, I think it's a Um, for for sure. Um.

Speaker 2: 43:03

I mean, even the little nuances class and you want to turn to they just say? What did you miss it out?

Speaker 4: 43:12

What was that? Yeah. What was that? Did you get.

Speaker 3: 43:15

That? Yeah.

Speaker 2: 43:17

I guess in a way you're missing But also you get flexibility, you've got partial online So yeah, it's got pros and cons. It's tricky.

Speaker 3: 43:27

It's really interesting. And I will tell you that, um, So when you look at results, particular of our students, we And um, we have other ways that we capture student feedback and the, they will constantly clamor for those sorts of things, right? We will have our, um, like our like older in age, they have things like that. And jobs nine to five and like can only come at night and all of those. And then we also have our online in the state or local in any way And, um, they clamor and say, like, we see all of these, these, these engagement activities that people can engage in, but there's nothing for us. And so it's interesting because institutions and you'll get a So we'll take you up on that. Right? But then when you look at the they're, they're asking for it. And so it's really, really really hard to do. Um, yeah, it's just really, I don't know how you balance all what they need. Right. And I think that's part of the complexity with the hybrid models as well, is like every one of these models that we, that we do a little bit differently, or like everything that we cater to means that we've got to add those resources. And again, going back to like higher Ed's under attack a little bit. Our budgets aren't growing. You know, they're so it's like, things that you want to do? So it gets to be like, it's, But it's also like just wickedly

Speaker 4: 45:10

Um.

Speaker 2: 45:11

I mean, you mentioned, um, know, I'm a bit rusty when it research methods, trust me. But you know, what's kind of the Um, so, you know, we were do you understand why students or two dropping out? And we tend to like, you know, quantum analysis or correlational data. But what, you know, is there a surveys or thematic analysis or, the students kind of opinions Or is it just hard to scale?

Speaker 4: 45:44

Yeah.

Speaker 3: 45:44

This is one that I'm actually Um, because this AI has helped, um, a roadblock here in ways that nothing else has been able to do. Uh, so, so, so I'm very excited about it, which I don't usually say about AI, but in this case, I will, um, because I like, we have like this giant backlog of qualitative analyses. And if you've done it, like, you it's so time consuming to do. And, um, when I am asked to use finite number of resources. Typically I'm asked to if you will. Like the numeric data, the, the Because people will be like, of trustees meeting. I need this number or, you know, They asked us these things. Can you give us these numbers? And it's usually always like, How many of these do we do? What how did was our conversion And it's all numbers, numbers, So in the past, historically, um, we've had this like backlog because I don't have a qualitative researcher on my, on my staff. I just have my analysts. Right. Um, so it's all about the, the So, but, but the AI has like Um, and so I'm very excited about this because I think, yes, we have started to be able to use this, um, and use those qualitative results a lot better. Um, one way that I, we are kind completed, but we've started to things and say, what are the We have these themes like what, but what are the students actually saying? And so we've been able to use help us code and look for the Um, we've been able to come up with some of those things to try and see if we can, number one, use this data that we're gathering. Like, I have a hard time. It's always kind of been a, it's bothered me that we gather this data and on students or from students and then we don't use it. Right? So number one, like, I feel like we're almost fulfilling this ethical thing, like where we're saying, okay, look, you gave us this and we are actually using it. But then number two, we're able to, to use it to say, like students are saying this, let's try it. Um, but so that's, I'm actually really, really excited about that. I think it's going to come back around and we're going to be able to use a lot more of that data.

Speaker 4: 48:11

Um.

Speaker 2: 48:12

It's fascinating, isn't it like Um, because so I used to do thematic analysis And it took, as you know, very, very long time.

Speaker 4: 48:20

Yes.

Speaker 2: 48:21

Very rich data, but takes a long course you're looking for themes on a large scale if you've got Um, so when you're saying using So would it be to try and like But I mean, is it, is it an they able to find the right doing it or.

Speaker 3: 48:43

Yeah, yeah, yeah. This has become one of my little Maybe this is my new hobby is training the AI to, to do the qualitative stuff. And it's kind of fun because there are things you can do, right? Like I do things where I ask it Like you're saying, this is a Okay, well, how many times did And then give me like your best And tell me how many like what? Like there's, there's pros and And you really got to train it. Like in the beginning, I was finding that the, that the tool was truncating. So I would give it a like a pages because qualitative stuff like seven of those. And I don't know what the cut And so like, it's, it's finding You just constantly have to be different than anything else. It's just a different way of Um, but it's been really fun to Um, we're, we've currently, kind of side by side where I committees and they've asked to So I've been like, okay, here Here's the data. And then kind of done my own scenes with the AI and then And you, I, I'm really impressed excited about it because I think analyze this like a qualitative to do it. But if you play with it and you the prompts and you have it be multi-layered conversation, I've I think it's fun.

Speaker 2: 50:13

Fascinating. So you're almost trying to train pick out emotive words like, is It'd be going, you know, analyze student's feeling about this. So feeling about attending And then it's got to try and perceive that or try and it's just fascinating.

Speaker 3: 50:31

I found that it tends to be like And so you have to train it to Like, I don't just want the bad You got to tell me the good So there's just things like But that it, that it seems like these tools are kind of they lean towards one way or the other. And so, um, figuring all that

Speaker 1: 50:50

Oh yeah. I think it's the, the art of, of science, I guess is probably the I think it's only going to I think what I've found is using it to not replace your work, but use it to, to validate and, and sort of use it as almost that that co-worker rather than sort of the, your, your sort of assistant. Um, it's it takes a while to get there, but once you start sort of getting those results, it's super rewarding. Um, I was thinking about the analyses and things and even out from that sort of thing can Like I think back to, um, one of the cafe was horrible and so no only going to get horrible food. And it was a bit like the university won't know that through. On a scale of one to ten, what If students are voting two, they're not going to go, oh, it's the sandwiches or it's oh, the coffee. But actually that one small sudden expanded over the thirty If five students go more often, And you've retained two of those Like it's, it's so it's so butterfly effect can be. Um, so to be able to get that sort of information, pairing that then with the whole, the big picture ideas and things like that, I think it's super valuable. I guess one, I guess sort of wrapping up sort of question, you've been through a huge project here over, over years to, to get to a point where we've got this massive data that we can use both qualitative, quantitative coming into the hands. Now we're getting to that, that moment, there's going to be are going to have two reactions. One is going to be, this is I can't wait to get started. This is super inspiring. The other side is going to be, I can't wait to get excited. Oh, this seems really daunting.

Speaker 4: 52:55

Sure.

Speaker 1: 52:56

What's your advice to, to those start this journey and go, where How, how do I get to that point?

Speaker 3: 53:06

Pick something. Um, and that seems really silly first analysis I run, I ran, more, um, was looking at, um, either participated in community sorry, community engaged And that was it. And it looked at like, is this And that was it. And I was able to and the reason So this is maybe like part two Pick something with a So I was able to pick that and, and in kind of with the stakeholder, um, show that information. And then like she presented it all sorts of, um, conferences And so it was, I was able to build the momentum from these small things because I paired a pretty basic analysis with the right stakeholder. Um, and then from there, other want that and I want that. And eventually it becomes, well, I can't do that until I get more resources. So there's a backlog now, right? And so it's like this grassroots I think if you're looking, if I don't know how to do right. I don't know how to do this or like, I don't know how to start it. I think that's what I would say. It doesn't have to be like a campus wide enterprise changing thing. Um, it can be one thing or two well placed marketing about it So it's getting the right stakeholders together that see the value in it and then doing your best to show them that value. So and that's that part of like I couldn't say, hey, you know, found this and here's the This was statistically I had to say, it looks like percent increase in retention if or something like that. Right. So, um, it had to be that like Um, and that is what I would say to start or don't know how to Like if you have the interest and you have the knowledge or somebody that has the knowledge to run the stats, then just start where you can and start small and like, but, and then like, tell the heck out of that story, right? Like that's the key. Don't just run it. You gotta tell the heck out of You gotta, you gotta really be able to translate that to, um, what I would say, like a naive stakeholder. That's, that's what I would say. Like somebody who isn't doing

Speaker 1: 55:48

Yeah. That's, that's brilliant It's that almost that Rome wasn't built in a day, um, sort of thing where you got this huge project coming up, start small, find that, find that ally that's going to take that work and shout about it to the right people or to, to as many people as possible. Heather, this has been Thank you so much. Thank you. Um, it's been incredibly Um, I've really enjoyed this I can see.

Speaker 3: 56:14

Yeah, you guys are great. Thank you.

Speaker 1: 56:14

That's smiling away as well.

Speaker 3: 56:16

I can talk about this stuff all This is so thank you for

Speaker 1: 56:20

Yeah, well, we'd love to have See what you know, other amazing Um, well, we're coming to this It'd be great to sort of hear, in, in about a year's time, twenty twenty seven.

Speaker 4: 56:36

Happy, lovely.

Speaker 3: 56:37

By then we should have like a full on training regime in place. So that would be lovely.

Speaker 1: 56:42

Well, that'd be great to learn Um, Heather, thank you so much. Have a great day.

Speaker 3: 56:47

Yeah. You too.

Speaker: 56:50

And that's a wrap. Thank you so much for joining us We hope you found the conversation valuable and that it gave you some ideas for how you can elevate your own student experience. Don't forget to hit that get your podcasts so you never Until next time, have an amazing