Learning Outcomes Leaders 002 | Tyler Walsh

Welcome to Learning Outcomes Leaders.

In this episode, we're joined by Tyler Walsh, Director of the Center for Higher Education Innovation at the University of Central Florida, to discuss how AI chatbots are helping students to proactively jump over administrative hurdles.

We explore operation insights and the university's massive shift from transactional advising to relational coaching model that prioritizes student wellbeing.

Let's get into the episode.

Transcript

Speaker: 0:02

Hello and welcome to Learning Brought to you by Genio I'm And on this podcast you'll hear professionals who can talk the there and done it. We'll discuss some of their projects, and explore the most helped to improve and elevate In this episode, we're joined by Tyler Walsh, director of the center for Higher Education Innovation at the University of Central Florida, to discuss how AI chatbots are helping students to proactively jump over administrative hurdles. We explore Operation Insights and the university's massive shift from transactional advising to relational coaching model that prioritizes student well-being. Let's find out more. Well, let's get into the well, thank you for, um, accepting to be on the be on the podcast. how we normally want, like to start and how we will start with the first question is we are super interested in people's origin stories. So how, what got you into the perhaps some formative education you to where you are today? Wow. Um, oh, I can go all the way Uh, you know, I think when my, asked me what I wanted to be, I, teacher and it was just that way Um, how I got into higher education specifically, I was teaching high school, uh, coaching and teaching high school for a while and, um, decided to go back to grad school, get my master's degree, and as part of my second master's degree program, I got connected with this group that was doing research on chatbots at higher education institutions. So, um, you know, twenty four AI enhanced chatbot was supposed seven one stop shop. You can get any questions Uh, they had been doing research on that for a few years and found that it was really working, right. It was bringing down some of the metrics that we wanted to bring down. It was increasing some of the So they decided, let's see what course level and we start Monday morning, you got this do you this long. Here's the link to go do it kind So I got connected with that group doing chatbot research as the graduate assistant, which, um, I kind of parlayed into getting a job at the National Institute for Student Success at Georgia State University in Atlanta, Georgia. Um, and then from there, uh, Uh, and so I worked with dozens of colleges and universities across the nation as a senior consultant there. Um, helping them both identify and then kind of, um, build programming and, and kind of new operations that helped dismantle some of the barriers students are facing here in, in American higher education and, um, University of Central Florida was one of my clients as I was a consultant there. And, uh, after working with to hire me on to be the director education innovation here at the And that was, uh, almost two So, um, kind of always wanted to be a teacher and, um, made my way into higher education to, uh, to not teach, but, uh, kind of be on the administrative side, uh, and help staff, help students. Really? That's fantastic. And when you, um, sort of sort of the, the, the chatbot What? Yeah. What was the sort of the scope Was it, was it things purely sort of around the logistics of a student, you know, how, what office do I need to go to to go to here? Or was this something that sort of became an ever growing sort of thing for different, different services and different access points for, for the university? How did that look? Yeah. James, it's a great question. Um, and you kind of hit the nail impact really the kind of, um, the, the, the way we like to Um, so the chatbot can do both It's a, it's a twenty four over student can say, I don't know I park today? And chatbot answer him. Um, hey, I have this hold on my in the College of business. What kind of majors are there? Right. They can kind of be proactive And, um, both the chatbot at Georgia State and now the chatbot at University of Central Florida, those are both designed using each student support unit or college or whatever department has information in the chatbot. Those are designed where that So my, my, my team manages the They send us all the information that they would want a student to know or like their kind of common FAQs. They'll send us those answers. Those answers live in a When a student texts in AI sees that word library and it looks like you're talking You want to know it's ours where What? You can access all this kind of So that's one half kind of the Students can kind of text into chatbot is the ability to So what we do now at UCF is we say, we just did this, um, a identified a list of students their account and registration So we got this list of students. We texted, all of them said, to be able to register. If you get this, hold off of If you don't, you're not going Do you need help getting this Reply one if you want help. Reply two if you're all good. Uh. And then all the students give it to our group of academic small kind of tactical team who and get some of the, you know, the right direction. Um, and they're able to triage students said they needed help. I called student, student picked staff morale because most of the picking up the phone is the part But if a student says, hey, I to call me, then they do you call them. Um, and I think our, our numbers were something like if you interacted with that campaign, right? We call that like a proactive If you, if you engaged with that registered at like seventy six engage with that campaign and registered at something like So it was about twenty twenty four percentage point difference just in saying, hey, please help me get this hold off of my account. So to answer your question, I guess, um, the chatbot really kind of does both of those things. There are ways that we that at the course level too. We can, um, identify students two or three weeks and they're the next Monday morning than every week or students who Hey, it looks like you missed You can only make up one exam or The kind of makeup extra credit Uh, we work directly with the students know where they're kind to get back on track or, hey, There's nothing you can do this Take the W try it again next Um, instead of taking the F right, in many cases taking the W's better. So, um, yeah, that's, uh, we're, research at, at UCF and we're chatbot research with the group twenty twenty one. Um, we are now doing it. It's now a national research study at Georgia State, at UCF and at the largest, one of the largest HBCUs in, in, in the US, uh, Morgan State University in Baltimore. Uh, so we have three sites and gateway English courses and see some of those DFW rates there. Uh, because those are the big student progression here in, in I the, the information you get must be super valuable as well to the rest of the institution, right? If you've got, if you've got frequently students asking similar questions, it shows you that perhaps they're struggling with this specific part of the institution. How do we work on that? Is that something that you feed into the rest of the institution? Um, Ideally, yes there are. There's about one and a half of us managing the, uh, the chatbot right now. So, um, we are currently in the middle of transitioning it over to a team that, that we've, we've had for a while, but we're kind of reorienting its strategic purpose into what we're calling a student success tools team. So they're currently that that operating our, um, advising So Ebe, uh, we call it, uh, midnight star branded for, for, for UCF. Um, but yes, the idea would be. You kind of take each unit's, um, data at the end of each year, right? So actually graduate studies Uh, hey, what how did how did graduate students engage with Nightbot? Nightbot is what we call the, the UCF Knights. Um, so what what questions were What time of day were they texting these in what day of the week? Um, those kind of things. So we, we actually compiled that report for them and sent it over to them. Um, and in the past, I think we've done that kind of unit by unit mostly as a, hey, here's all the questions your students asked. Are these the answers that You were okay with twelve months that still right? Um, and, and no, we have not able to then kind of collect and sure that the university Um, ideally, yes. And I like where your head's at. Uh, but, uh, that's going to have right now. Well, at least it's, at least I think it's, it's just like, useful thing for students, but in and just take that little bit bit of support? I could chat about this, uh, chatbot for the rest of this session. Um, but do you have a specific of that you want to talk to talk Why did you want to pursue it? Yeah. Um, I could talk. I didn't even think about the chatbot as the project I'm most proud of, but, um, I am really proud of that, that research to, um, and research with a, with a lowercase r, this is not capital R, you know, uh, research that, that, that we're going to publish. Um, but yeah, I think, uh, you about, um, you know, kind of and using it to inform what support unit or at, at UCF, you cultivating a healthy and wellbeing ecosystem, right? We kind of view all of these different student support units as component parts in a larger whole. Um, and so one of the projects most proud of is, is a project And it's just that it's, um, developing a data collection system in which everybody is inputting data in a standard and systematic and accurate and consistent way. Pulling that data out, um, cleanly and rendering it back to that unit in a way that they can, uh, you know, they can use that data to inform their strategy, their kind of daily, weekly, monthly decision making, right? This is all everybody's shaking Yes, this, this makes sense, This is simple stuff. Everybody's doing that. Well, it's not true. You know, I would say we're we're probably one of a handful of schools across the country that has the student level data available to us that we have now. Um, so, uh, that's, that's one that I'm, I'm, I'm very, very proud of, um, a little bit of context to that, I guess, uh, you know, we, underwent a couple large restructures in the past two years. We're, I guess three years. Um, UCF as a whole restructured into the division of student of in the middle of twenty twenty twenty three. SWB so the division, student SWB restructured all of our advisors out of the colleges and centralized them into a new model called Academic Success Coaching, which is a different strategic vision than just kind of advising, which we consider a little more transactional, right? Get registered for classes, make Make sure you're there. That's it. Come see me once a semester. You're set. Success coaching is a little bit It's designed to deliver kind of mental health support. And all we have dedicated staff But you know, this kind of sense Come see me for fifteen minutes Let's, let's check in. Let's make sure that we're doing You know, my boss asked me, hey, if we're going to be delivering holistic care, we don't have holistic data. So how do we know? You know, we're only looking at one slice of a pie for the student. How do we kind of get a better picture of where each student is at? And that's where Operation able to provide that data to our have more data, informed Hey, it looks like you are That's going to be a really, really critical course for your medicine degree. But students who also struggle in biology can switch over to bioengineering. And that's, you know, they have Not directing students what to do, but giving them the information they need to make the most informed decision on achieving their, you know, academic career and kind of lifelong goals. Um, so both to get the data in hands who are our kind of There's all kinds of, but we're towards success coaches. And then also to get the data Right. If, if our president comes down investment in switching all the and calling them success coaches And, um, it's a, I think we're now operating the largest success coaching model in the nation. What are we, what are we getting Right? How many times did we meet with What kind of students were they? What were they talking about? Um, that's the kind of data that never had access to before. So I can tell you how many with last week and what those those sessions were. And if that coaching session was had sent out, or if it was the proactively scheduling with that percent of a ASC's roster that So all of our ASC's have rosters of students that they kind of keep from freshman year to, to, to graduation. Um, so this is all really helpful data, both for our understanding of how we're impacting students. Um, and then, you know, also the, the, the frequency of that impact, um, and our ability to give ASC some information about their students that say, um, or that help them, um, support students in making the most informed decision for their goals. That's, that's, that's really I think we often have a lot of students experiences in college of so many different parts of determine whether or not a I remember as on a previous somebody and they mentioned they university and they mentioned look at on student intake is a student comes from because particular high school, maybe actually as good as, as, as And so those students are sort higher up the, up the chain for, things like that. And those small things that you about at first thought of what but actually something as simple when they were thirteen and how of college can be so big. I guess what was sort of has been some of the more maybe not surprising maybe, or, or sort of interesting pieces of data that you've been able to collect and sort of go, this has actually had a major impact on on students and their success and things. Yeah, it's a good question. So, um, we started this in And so this fall semester is the first semester where we've been able to collect kind of what I would consider clean and valid data. So in our fall semester just So we, um, we've got, you know, fourteen weeks now of, of data collection. Um, so I can't, I can't, you know, um, speak to any kind of causality, obviously. Right. Or even, you know, strong Um, and we don't have grades impact data on all of this. I can't wait to get, hey, if is, is, you know, meeting with an ASC for fifteen minutes twice a semester more impactful than meeting with an ASC once for thirty minutes. You know, questions like that is what is what we're kind of asking. Right? Um, but, uh, one of the things um, we're really, we, we also of, um, organized and, and strategy to students this insights was right. If, if the kind of underlying assumption here is that the more students engage with coaching, the more likely they are to succeed. Then how do we get more students Well, let's email them and say, hey, you should come meet with me. So we had this kind of email track really well and all this. Um, and one of the things that we found so far is that about eight percent of our twenty eight thousand total sessions from this semester were due to a student receiving a campaign, clicking on the link in the campaign and scheduling from that link. So that's a pretty low percent. I was not I was expecting something higher than, you know, kind of eight percent of students reacting to that campaign. We're getting more engagement You can look at some of them. The open rates at sixty seventy You know, some of them have from that campaign. But you look at campaign by campaign, you're like, oh, that's thirty here, there's sixty there. There's, you know, fourteen You think about it in the grand appointments across our one that's, you know, eight sustainable strategy, right? So that data point in and of itself has made our has, has, has gone back to our leadership team. And now we're looking at that the way that we're campaigning? Um, or should we kind of switch the availability of win campaigns? So, so there's all these little behind it that this data is now well, that wasn't the greatest Let's let's rethink this one. And obviously getting feedback That's been really, really been sharing this data back with know, getting a context for what And, you know, why are we seeing And, um, so we've been really ability to kind of open up new strategies to, to, to, to get new avenues of conversation you know, leadership, staff, and being part of this. Um, so that's one data point Um, and then we, obviously, know, four or five kind of cross So, um, if we're thinking about students, we can isolate that what GPA ranges are there, what And so we can kind of look at of becoming a stronger look and see, okay, how many What GPA ranges are those in? What year are they mostly fourth need to shift to focusing on You know, those kind of Um, so I don't know if that student level data, but, um, Yeah. No, definitely. Definitely did. Thank you. And so all of this seems like, you know, collecting a lot of data, there's a huge amount of data points that you need to collect. What was one of the most challenging things in getting something like this set up across the entirety of the campus? Um, that, we work in higher Uh, you know, I, I guess I guess I've never worked outside of professional career, but one about, you know, how, how higher loath to, uh, change, right? And or, you know, update, And that's really, you know, higher education in America is it's set up over there the same governance structure, right? We can't do anything without Um, that's not necessarily true for this project, but that kind of attitude, I think, or at least perspective does kind of bleed into a lot of what we do as administrators. Um, so one of the ways, one, one think was right at the start before, my boss kind of asked if my boss comes to me and asks did we have last week? I can't, I don't know. You know, like, that's a pretty simple question I should be able to answer. So can you, you know, you take your team and go, go help me answer that question kind of thing. And so what we did is we started pulling all the data that we had already, right. But all of that data was being coaching team in different ways So the data was not defined in a It was not defined in a standard So essentially what I did is I that showed how many coaching said, you know, this would be defined the same way by actually draw some insights from So I think the, the most Making sure that there was a common understanding that the data that we currently had was not sufficient. Right. And so, and kind of explaining So really kind of getting into is not valid. What does that mean for us as What does that mean for staff? And how can we all work together to get clean and valid data that that kind of continual, I guess, reminder of why this is important to have and what and like why we're asking you to change just a little bit of how you input stuff on the screen is going to have huge dividends, right? So getting, I guess getting the buy in was, was one of the most challenging things. Um, one of the ways that we boss, he's the, uh, senior VP at UCF and he's also the, uh, So, um, he's, he's, he's a nice have on your side, right? Uh, and so one of the things that we did is we essentially developed a team of from Ryan senior VP, another AVP was in there as senior directors in there. A couple direct, four or five of made this, what I have termed structure of empowerment, right? If the if the person who can make the decision on the change is not in the meeting, then everybody in the meeting is just discussing change. They're not deciding on change. Right? So what we did is we held kind of biweekly meetings with all of the people who could decide on changes in the room, and we decided on changes every two weeks until we got to a point where we were like, okay, this makes sense. We can now roll this out. And we had all the people in the roll it out. From day one, we had my data So when we got to the point of collecting data, I didn't have to go explain to my data analysts. Hey, this is what we want to from from the jump. And now they're coming to me Oh, hey, why aren't we Didn't we say that this was part Oh, yeah. I didn't even think about that, you know, so kind of having all the people in the room, specifically the leadership who can make the decisions on moving everything forward is how we kind of overcame the challenge of both buy in and also momentum, right? Bentham. Um, and we're now, you know, road kind of for the first time. Um, we have spent the last two of the findings, some of the We're not calling it findings semester's worth of data. Um, but sharing that back out staff, right at kind of large Um, and I suspect that that the yet to come. Right. We're, we we kind of have, you we've never had before. So, um, they're direct. So directors of each one of the teams can now kind of use this data to have constructive conversations with, um, ASCs about, uh, you know, how many students they're seeing and how many students they may not be seeing. And, you know, uh, all these Hey, what do you, what did you sessions in, in one week? How did you set up your days? You know, did you have only How did those go? Did you feel burnt out? Like we're trying to figure out of a happy, consistent medium amount of students every week Right? Sometimes, especially in higher education, it's like, right when we come up to midterms, we go from thirty or forty in a week to ninety to one hundred in a week. And we'd rather pre-stage that everybody a more consistent flow So I think that that that's more challenging parts is kind um, getting in front of those kind of conversations. I, I love that, that you right people in the room to make And then, um, because so often you could sit in the room having great ideas, but you need that one person who can, who can tell you if that great idea is viable, right? Yeah. If they're in that room, they're They can go, okay, yeah, let's Let's do that. Let's do that instead of having to move that a great idea and then go over to them separately and go, hey, we spoke about this. It works like this. You've just got those different And then trying to articulate it in the same way that you did in that room isn't going to be the same thing. So it's so important to have room, um, just move stuff away higher education as well. Um, having all this data is. Is super, super useful, um, for responded to it. Um, very positively so far, very Uh, we haven't showed him any of the bad stuff though, you know, so. No, um, we are, uh, I think team and, and I think everybody, that are, that are part of the don't, I'm surprised maybe, um, we've just never had this So like even six months ago, we didn't have the ability to, to answer some of the questions that, that we can very simply answer now. So that's been really for me and my team to kind of. Continue, continue on this path. Um, and you know, as we're Um, you know, over the summer, leadership and, um, Nacada. Uh, which is a advising kind of Um, and we were just sharing for this fall. Right. We were kind of six months into the building and planning of the foundations of how are we going to ask our advising community to change the way that they're inputting the data about their student sessions? Um, and that's where we were at And all I had was the vision for how we were going to get all this data. Over the summer, but even sharing that vision out with them and, um, everybody was very interested and very excited about the potential of what we could do. And I can tell you now that we and, and then some. So our plan very much in twenty sharing this out with the rest so that we can help others get Right. That's kind of our, our whole to the front. I've always wanted to be a want to do. So, um, as we learn at, at, at responsibility to, um, help because we learned how to do Uh, and, um, if we can do it at largest universities in the it anywhere, right? We're doing it at a scale of sixty thousand undergrads, ten thousand grads. Uh, if you can do it there with success coaches, then, um, you kind of, uh, you know, operation thousand students and, you know, advisors kind of thing. So, um, we feel like since we can, we feel like when we figure out how to do things at scale, then, um, it's a lot easier to, to bring to the rest of the field who maybe aren't at our large of a scale and we can get to sample size, you know, statistically, statistically significant sample size very quickly. So, um, all of our, kind of like level chatbot research, we're national pool or that, that kind So, um, that's our plan right now is to, uh, take what we've done and help other people, help, help other institutions figure out the way to do that in their context. It's not going to look the same as what, as what we did, but, um, how do you do it with the resources you have, the folks, you have the student type that you have. Um, because yeah, everybody's been really excited about about what we've been able to do so far. I really like that. I think it's we, we work a lot with data as an organization as well. And I think it's we've got on a similar journey ourselves, right? We, we're a company of only and fifty people, but we have single day internally, conferences, we're chatting with and it can sometimes be very passed along, almost like broken I heard about this from this quantify if that is something I heard this stat and it, it can be very difficult to, to, to pull all of that information together. And we've, we've sort of gone on a similar journey in that sense of how are we going to quantify this information? Where can we store that, create Um, for us all, and I can, I can in a test from from my own personal standpoint, in a company of a hundred, that having that centralized knowledge base is so useful because it saves everybody time, um, in finding information and feeding things back and getting things changed. So doing that on a scale that you're talking about of one hundred advisors in one department and then eighty thousand students on, on top of that, you can see how very quickly that that becomes so revolutionary and it creates that, that two way, um, two way street between data coming in and then the outputs going the other way and the changes, um, that need. So it's, it's, that's, that's I think when you said about, you know, that next step being we want to share this with, with the rest of the community, how, what's, what's the plan for the, for the, for the sharing of that? Have you, have you got things year or so to, to start Um, I wish we already had all We do not yet. Um, our first step is to kind of share everything that we've done with our executive level leadership. Right? So, um, towards the end of this of weeks, um, we're going to put what we did from January first of get this ball rolling. And this is where we're at Um, and then from there, definitely try and hit all of the major advising conferences next year and just kind of share out. Um, this is not only the, the, the results that, that we've seen, and here's some findings that may apply also to your students and your institutional context. So if you're seeing this much of did to, to see more of that So those kind of things, but empowerment idea too, right? Not only these are the kind of Strategies that we've deployed that have helped us have more of an impact, but also how do we even get here in the first place? Who was in the room and what did we take to get there? So that's definitely our plan is how do you better impact your design a system that is less That's more efficient, more Um, that's streamlining things for, for both students and staff. Uh, and you know, what does that You know, for the first three We decided on which metrics we And we did nothing other than say, do we want to track this one? Yes. Do we want to check this one? No. Okay. Why do we want to check this What are we going to do with If I track how many unique you going to do with it? ASC director, if you can't tell that insight and deploy it in a to track it, right. So we're not tracking everything Um, we're specifically tracking certain things that are going to get us those cross sections of kind of, uh, what's the LinkedIn word KPIs or something like that. Right? Or what is the, what are those cross sections we want to see movement on? Because our goal is right is to increase our four year graduation rate. Well, if we implement a new strategy, fall twenty five, we're not going to know if that strategy had even any kind of correlation until fall twenty nine. And we can't wait until fall So what are those leading Right? So all these kind of ideas, um, our, in our rollout. And I think, um, just to touch on what you were saying there, James, one of the most critical things about that kind of, you know, common data set, so to speak, is not only, you know, everybody can kind of see all the same data, but it becomes that single source of truth, right? So one team isn't looking at, sessions and one team, I've had from different places and they So having that kind of really does help everybody where we're at from that, that, I, um, I think about something talking about a campaign, which, Um, and people view data as this data so we understand what But I like data also informs currently or what you were doing So in the past, if you didn't doing the same thing not knowing Um, from the data which you've got out so far, is there anything that stands out to you that you can remember that you were like, we've been doing this for so long and it's not been working because now I can see it isn't. Um. Yeah. The best example is the one I've I think the, the, the percent of sessions that are held kind of as a result of campaigning students. so here's, here's a interesting So one of the things that one of, one of the strategies that we're trying to deploy with success coaching is, um, you know, something called follow up appointments, right? We want to make sure that or three times a semester, not Just check in with your ASC fifteen minutes a semester, maybe twice a semester, that kind of thing, you know, two or three times over a academic year. And one of the ways that we want to track how well we're doing that is how many of our future Sure appointments are being scheduled by the ASC versus by the student. So if I'm in my first session of session, ASC says something time before the end of the term. Let's go ahead and get that I'll schedule it for you. The number of sessions that are scheduled by an ASC in the future, versus number of sessions that are scheduled by a student, is one of the kind of proxy measures that we're using to check on follow up appointments. That's not always true because meet in December and student do it right there. So it's not it's not an exact But, you know, we're looking at something like as our starting point. And again, just this is just Um, eleven and a half percent of next thirty days were scheduled That's eleven and a half percent So we have two thousand seven in the next month and two months days, eighty seven percent of by the ASC, and three months are scheduled by the ASC. So we can kind of read that as, okay, we we've really been pushing this type of strategy just in the last couple months, right? So ASCs are starting to apply February appointments, not for really make sense. Right. But we can kind of look and see This next month, this kind of where ASCs may be applying that and a huge jump from eleven to ninety eight percent as we three kind of thing. So that's a really interesting And that's all it is, is This is kind of the first marker in what could potentially be a trend. But I think that's the other thing that we've been really careful of in rolling this out is we haven't rolled this out with, oh, this is the data that we're seeing and this is the the benchmark. And we're not hitting that. That's not what we're doing at We're saying all we're doing is clean and valid way. We're going to share that data going to make collective makes sense as a benchmark. What then makes sense as a goal You know, we're not trying to do any of that without gathering a whole semester's worth of data first, but that, that, that would be a, a interesting, you know, we maybe more of a, of a, of a success in a, in a potential strategy that we want to deploy rather than a, you know, this one's just really not working. Um, one of the ones that that's really difficult for us that, that is a strategy that currently I would suggest is not necessarily working is, um, you know, we, we meet with a lot of three point oh and above students, right? 3.0 GPA and higher students. Um, our goal right is to start point five to three point five, kind of students. That's one of those cross sections of the data that we say. If we're meeting with more two students, we expect our four Right. That's kind of a leading Now that's really difficult at students are three point oh and Something like sixty seven three point oh. So how do you go tackle that student body who's not. Those are typically the students phone, who are not going to And if they do make an appointment, are going to no show, right? Uh, and I'm not, you know, I'm not trying to stereotype any of those students. I'm literally looking at the If you switch from three point oh and higher, no show rate, it's something like eight percent. So they've scheduled a meeting, that coaching session. If you switch from two point oh to three point oh student, that no show rate jumps from eight percent to like twenty two percent. So maybe we are finally getting But then a fifth of the time we're not able to actually have that session. And there's a number of These are probably working students, maybe first gen students, right? Um, parents, all this kind of So what are some strategies that Again, trying to be a, a, a What are some strategies we can deploy that help that kind of student out? Maybe we hold sessions in the days a week. Maybe we have kind of more like know, there's maybe we go meet know, like, what are some of the kind of murky middle? Um, I don't know yet because we don't have all of that data in, but, um, that is one of those things that, that, that we're looking at as a, We don't need to be meeting with our four point oh students three times a semester. We need to meet with them once. We need to meet with our two semester kind of thing. So, um, that's going to be a really interesting data point that we want to continue to track that. I would say right now we're, we're not where we want to be on. Yeah, that's, that's really Um, insight. And I feel like I'd be really proactive booking of those over the next couple of years. Like you said, I think my staff know the ebbs and flows of get that in. Whereas a lot of the time I know student, I didn't need help needed it immediately. And it's, you know, so I'm not two months time. I'm, I'm sure I'll be fine by It's only midterms and midterms arrive and I need a meeting straight away. but being able to then have that students, well, two two point Other students in the past have And what we know works is this it's going to be so valuable, not just internally, but in, in those actual communications with students to say, we have met with a thousand of you before, and this is what, you know, this is the playbook that we've found. Here's the stats. It's it's going to be so powerful for you to be able to have that, that in the future for, for that internal and external. I think it's brilliant. Yeah, that's exactly right. so, yeah, um, we can give that data just like you were talking about. James to be able to email to a Hey, in week eight, I am going to have zero availability because I've had zero availability in week eight for the last two semesters, but I have lots of availability in week four. If you want to come see me in time and you're going to get all for week eight two in, you know, pre-stage some of those spikes I can't my, I can't see my ASC, I don't, they don't have any availability. Well, just like you did James, you know, as soon as midterms come up, everybody tries to see their a C in that week and there's only so many hours in a week. Right. So I did the same thing except I did, you know, I didn't even know I could go to my advisor to see them. You know, that's, uh, so at least we're doing better than that, right? Yeah, I absolutely, I think we organization, when we. You know, when we're trying to of our tool, right. Be using different messaging and this amount of students improves don't you get in on this? It's going to resonate more as a message than here are the features. It's, this is the impact that that same sort of thing of we this in, in a session versus give you as a student. I think it's, um, it's super, I guess as we, as we come to, to sort of the end of the, of the podcast, um, it'd be really great to just sort of hear if you've got, um, one piece of advice, like let's just say one piece of advice for anybody listening today that is looking to, to roll out something similar. How, how do they get started? What's that? What's that sort of key thing that you would, you would say to them. Oh. Just one. Um, like I'm some kind of fount Um. You know, I think two things I think they're connected. One is, um, you know, start, start to ask a really simple question that your leadership may ask you at some point, right? Our simple question was, how have last semester? We can pull the data. That's that shows us how many appointments summaries were submitted by people that we call success coaches. But that is not the definition Some of those might have been emails, some of those might have been phone calls that never went through. Like there were so many different ways people would be submitting this form that was counting towards what we would have previously. You know, so figure out a really or figure out a really simple answer, and then go see if you the data you currently have it And then the second thing is, is when almost inevitably you can't get your data, people, wherever they sit in your organization and bring them on the journey with you from the start, because none of this happens without our data analysts being able to collect clean, consistent data out of two or three disparate CRMs, bring them together somehow in their magical data land, and then regurgitate them, render them back to us in a dashboard, in a power BI dashboard, right? If you if, if those folks don't really understand the strategy behind why we're collecting this data, don't really understand what we're trying to render the data into so that people can use it. If you have to go explain that to your data analysts week after week, and they're not just in those conversations. Hearing it, it's a lot more you're going to result with see and not things that, you people who are dealing with Not that the things that data want, people want to see are bad. Uh, they're normally incredible. They're normally just, uh, not as readily usable as, um, So that that's what I would say really simple question that you data can answer it. Um, if it can't answer it or if you think it can answer it, but a data person comes in and says, no, actually, that's not what this means. Um, then keep those people that kind of vertical structure Just trying to answer that I, I love that advice. I think, I think that would get themselves a lot, right? If I if I go out and just try and search for this one, one answer. Oh dear. I cannot find the answer to What things do I need to put in It's like sparks that urgency I just want this one answer. I can't get it. Um. Super useful. This has been a very enjoyable Tyler. Um, I think so too. Amazing. Well, we would love to have you on in a year's time and talk about where the data is at and what's happening. That would be that would be very This time next year, we can I'm sure you would have implemented lots of cool stuff because of the data. I assume so, yes. Um, and we can talk about your, conference run as well. Yeah, yeah, hopefully I'll have and, uh, we can actually, um, missing piece from all of this We don't have our academic outcomes for this semester yet, but we'll have those in a couple of weeks. But the idea is to attach, you this term GPA to each student most impactful thing that we did You know, that drove a Now that's just one slice of the We're a holistic model too. So how do we start to collect engagement data or, you know, student sense of belonging or meaning, you know, those kind of things. So we're also adding some of more holistic data set to this. But, um, last piece of advice I guess is this is a phased approach, right? We're not trying to tackle all We're trying to tackle this part Next semester we're going to add that, add a little bit more. Um and so yeah, hopefully in a impact data in here too. And we can really talk about the strategies that are working and the strategies that are most impactful. Awesome. Well, thank you once again, and Um, and wish you a good day. Yeah. Thanks, Scott. Thanks, James. Thank you. And that's a wrap. Thank you so much for joining us We hope you found the conversation valuable and that you gave you some ideas for how you can elevate your own student experience. Don't forget to hit that get your podcasts so you never Until next time, have an amazing