Measuring Embedded Career Learning

What does it take to embed career development learning across an entire university—and prove that it works? In this episode, Jeremy sits down with Julieanne Cutrupi and Eva Chan from the University of Technology Sydney to explore their multi-year journey from an opportunistic, opt-in model to a curriculum-embedded approach spanning 73 degree programs and 135 subjects, reaching approximately 33,500 students. Together, they unpack how a team of just 4.6 equivalent career professionals built this scalable career ecosystem, developed a framework for measuring its impact, and generated compelling evidence that embedded career education significantly improves students’ career understanding, clarity, and confidence.

Transcript

SPEAKER_02 0:00

Welcome to the Career Ecosystem Era and Higher Education podcast where we explore the systemic changes colleges and universities are making to ensure every student receives the career learning they need to succeed. My name is Jeremy Padani. I'm the CEO and founder of the Career Leadership Collection, a consulting company dedicated to helping higher education transform their career ecosystems. Now today I get the distinct pleasure of sitting down with Julianne Katropi and Eva Chan from the University of Technology, Sydney. Julianne is the manager of UTS Careers and Eva, the career development manager. I first met Julianne in 2017 when she was a featured guest on season one of our online show, Everyday Innovators in Career Services. Now I must say she was the first and only guest for eight years who literally made my draw job, my jaw drop on a live episode. Her innovative approach to career services has truly shaped the landscape of careers at UTS, and in my opinion, across Australia. Flash forward to 2025, where I was in Sydney as the keynote speaker for the NAGCAST conference. This is somewhat like the Australian equivalent of NACE. And I sat in on Eva Chan's presentation on where UTS is today with its approach to embeddable career development learning. Well, the eight-year drought was over because my jaw dropped again. And the invite had to come for them to be on this podcast. In 2025, career development learning at the University of Technology Sydney is embedded across 73 degree programs and 135 subjects, reaching approximately 33,500 students. This initiative was delivered by 4.6 full-time equivalent careers consultants. Today, they're going to unpack what a multi-year phased approach to embedding career education within the curriculum looks like, how they're measuring it, and what it means for students' career clarity, confidence, and knowledge. Julianne and Eva, welcome to the podcast.

SPEAKER_01 2:32

Thank you, Jeremy. That's such a generous introduction, and it did make me make me chuckle. I'm a big fan of your work and have been for some time. So it's excellent to be back and to share a UTS update with you. So thanks again for the invitation.

SPEAKER_00 2:47

Thank you, Jeremy, as well. It is our honor to be able to share some of our experience in the past couple of years with your audience.

SPEAKER_02 2:56

Yeah, well, looking forward to the conversation. And let's set the stage for everyone. Julian, perhaps you could tell us a bit about the UTS context and the journey that your team has been on? I mean, when people hear that career development learning is embedded across more than 70 degree programs reaching over 30,000 students, what should they understand about the scale and intentionality behind your work? And moreover, what what made you realize that the old model wasn't enough?

SPEAKER_01 3:32

Yeah, yeah, happy to share. So um conscious we've got a global audience. So UTS is similar in a lot of Australian institutions, a little large. So we do have around 50,000 students across nine faculties. So for us, scale is always being in front of mind. It's something we have continually tried to conquer over the years. And when you hear it's across 70 degrees and 30,000 students, I imagine everyone asks that how, how, how have you reached that point? And then I'm sorry, there isn't a silver bullet answer to speak, but it has been something that's been really intentional over a number of years. Um, it hasn't happened overnight. Um and I guess it's been irritating. It's irritated. Sorry, it it's it's evolved. So um we've designed our approach to really sit within core subjects. Um, we've moved from electives and align with key transition points throughout the student journey. So why we've taken that approach is it means that our consultants can actually tailor and contextualise the career development learn into where the students are in the curriculum and what they're currently navigating at that point. So it doesn't feel like an add-on or um something they have to go out of their way. It's really um connected to where they're at in their curriculum. And that focus really came from us recognizing that although our past models were great and we were doing excellent work, there was no doubt there. It sometimes did feel opportunistic, and um our students were still needing to opt in to especially our co-curricular work, and therefore we were only really reaching the students who were already engaged and missing lots of other students, which to us wasn't good enough. You know, we essentially came to a point where we didn't want students to miss out because of a timing issue and something conflicted in their diary, or they simply couldn't reach um our services outside of what was already on their plate. So um yeah, we we took this approach thinking the one thing that all of our students experience is the curriculum. That's the one thing that is every student has in common. So we we're like, if we're gonna be serious and we really care about equity, scale, and impact, which we we say we do, and if if we truly do, we have to focus on the curriculum and then build something that's consistent, embedded, and designed to last. So that's I guess a little bit of our background and where we are today.

SPEAKER_02 6:17

Yeah, really thoughtful and and intentional over time, like you said, to embed into the context that you observed that would work so that you could move the needle on your values. I love it. And and you started to build a new model. And so, Eva, perhaps you could walk us through some of the phases of building your career ecosystem over the years to get to the point where career development is embedded for almost every student now.

SPEAKER_00 6:52

Most definitely. As um Julien was saying previously, our model was a bit more opportunistic. And um, our faculty engagement strategy has really started to evolve in the past couple of years. When we think back to around 2023, this is where we are quite intentional looking at where we are delivering within the curriculum and want to um bring in a bit more of a structured and a scaffolded approach. So in 2023, we moved from focusing on elective subjects and more of an ad hoc delivery to really focus on the core subjects. This is for us to try and provide a better scaffolded experience for our students and minimize the duplications in them coming across with the career development content. Also in 2023, we looked at how do we deliver more flexible um methods of delivery. So, for example, for subjects that are quite large with enrollments that are a couple of hundred up to over a thousand students, we um brought in the train-to-trainer model. So, this is when we develop content that is tailored, we develop lesson plans, and then we run trainings for academic staff and the tutors for them to deliver those contents within the curriculum. Ultimately, out of 2023, we realize our major goal is really having one touch point per year per student within their degree program.

SPEAKER_02 8:18

Will you soon be searching for your next career services leader? At the Career Leadership Collective, we focus exclusively on executive searches for career services leaders, and our network is second to none. We've helped place transformational career leaders at elite liberal arts schools, large research one institutions, top 50-ranked schools, business colleges, mid-major schools, Jesuit institutions, and many more. Our annual leadership development cohorts enable us to leverage our nationwide network and quickly deliver high-quality candidate pools. If you're in a transition and ready to bring in a leader who can elevate your career ecosystem, let's talk so you can benefit from our network of the most innovative, trusted leaders in the field. Visit us today at Career Leadership Collective.comslash search or email us at search at career leadershipcollective.com.

SPEAKER_00 9:17

It was 2023, and then in 2024, we realized apart from educating our students, we really need to try and improve how we communicate with our faculty staff as well. So we set about to explore what are we trying to embed within the curriculum. Our team really sat down and looked at what are the different topics that we've covered so far. What do we want our students to get out of when they come to classes with us? So we out of that, we deliver a uh we develop a career development framework. It incorporates 10 components that are essential in career development learning. And that framework with the 10 components, we match it against the different years of studies. First year students, middle year students, and then final year students, which in with each of those components we ask what are the questions students are likely to ask? What topics do we want to embed within the curriculums? If we were to embed what activities we can use in class, in person and online, and also if there were assessment items, what assessment items would we recommend our academic staff to use? Apart from that framework, we also start mapping where we are so far in the faculty, looking at flagship degree program within the each of the faculty, looking at where we are across the years and across the different um majors and subjects as well. So through those, we were able to look at very quickly what are the best practices within each of the faculties, where are the major gaps as well. So from that, that helped us to really present uh a better narrative when we go and speak to our academic staff members, looking at what we're doing well and where we can partner even more with them. So that was 2024. And then in 2025, um UTSS with many other universities in Australia. We were going through a bit of a um, I guess, university-wide restructure. We understand the major focus in 2025 is really for us to look at how do we profile our work and how do we actually provide the evidence on impact of what we do into students' graduate outcomes and into their career development learning. So that was the focus for 2025 that we looked at. How do we measure and really build a narrative around our work? So you can see this is a I guess multi-year journey that we've uh went through how we um our faculty engagement strategies really evolve over time.

SPEAKER_02 11:46

Yeah, and if I could use my English major alliterative background and say you went from meeting every student to mapping the gaps to measuring what actually matters. And uh I think I think it's uh it's a really brilliant approach. And um I'm glad that you got to the measurement phase because a lot of times we can end on, oh, this was a cool program, and and and the the impact itself isn't as effective. So let's talk about measurement a little bit. And you you use the phrase in Australia, you use impact measurement, meaning uh measuring the actual results, the impact on students that that you're having. So, Julian, could you walk us through why does measuring impact matter in your context?

SPEAKER_01 12:39

Yeah, yeah, happy to. And I guess um, following on from what Eve is just shared, we'd been in a fairly fortunate environment at UTS that we were um supported by senior executives, people were very sort of happy to open doors for us, and um as we mentioned, it's a practice-oriented university. So um academics are very happy to work with us and connect and engage with us. But I guess the last few years with that restructure and we noticed a shift. There was new leadership who again were impressed by our work, but we didn't necessarily have that impact that we could demonstrate we were making. It was more reliant on anecdotes and you know, our case studies of good work rather than this is the difference that we make. And we we noticed as that um as that organizational restructure was coming along that we we really needed to focus on that. And I guess that was some of some of the driver, also of you know, we it was the next, I guess, stage of um our growth, but um, yeah, it really did help us focus on on the impact and get why why does it matter? I think it it matters in um in those hard times, but also when we are meeting generally with our academic colleagues and sharing how we work and the difference that we make for students, it means it's much clearer and easier for us to share how we work and the impact that we make. Um, and uh through that alignment, looking more at you know, student success and outcomes rather than just did we support the students that came into our office, as you as you mentioned. It's um yeah, having that evidence to really help people understand where we're reaching and the difference that we're making through our work.

SPEAKER_02 14:31

Yeah. And I remember being in Sydney and listening to some of the ways when you were presenting this, and I I came in the room just a little bit late, and I look over at my colleague Nick Catton, and he's taking pictures. And I don't ever see him take pictures of slides. And one of the junctures he was taking and he looked at me, he's and he like waved me over, like, come here now. Like this is really fascinating. One of the things that he was taking pictures of was um uh you have a five-tiered impact measurement framework. Uh, can you tell us a little bit about that framework?

SPEAKER_00 15:10

Yes, of course. Um with our focus on um measuring impact in 2025, the first thing we said about um for our team to really reflect on is how do we actually measure and evaluate our work within the curriculum. So we sat down and looked at the what the measures that we've done so far and uh look at um potential measures that we can put in as well. So out of all those um ideas that we've generated, we managed to put it into a five-tier model. So when you think about the T1 and T2, they are more the traditional measures that a lot of us would use, capturing the level of reach and the level of our activity. So, for example, there are traditional measures on how many students have we engaged within the curriculum, how many subjects and courses do we embed our career development content into, how many hours of deliveries do we do? So that's the most basic level in Tier one and tier two. Then in Tier three, we start thinking about there's a lot more that we should be reporting on um other than just the traditional stats. Um, we started thinking about engagement and feedback. Um, we of course collect student and academic feedback out of the sessions that we run and the content that we embed, including quantitative and qualitative feedback, but there's a lot more to it as well. So, T3, we also try and capture the level of participation and engagement. So, for example, if students are, are they actually engaging in our content? Um, are they participating in our online modules as part of the curriculum or not? T three, uh T4, sorry, we started looking at behavioral and learning change. The major question is are students actually applying what they are learning into real life practice? So we start looking at um can we put in a change measure in terms of their career readiness? Can we start reporting on how many subjects and courses have career development learning assessment integrated into those um the subjects and courses? How well are students doing in those um assessment tasks? And then Tier 5, which is the most important one, it's for us to start looking at how much are we as a team influencing curriculum design and redesign, how much are we engaged in the um re-accreditation process of degree programs? Also, how much are we contributing to the graduate outcomes? So for us to pop all those different measures and ways of evaluating our work, what we can see is there's no one single measure that is enough. Rather, what we need to do in terms of what measures are we do uh going to implement, it's really considering what narrative of impact do we want to build, what story do we want to present when we are presenting our work to different audiences. This is for us when we work with our faculty partners, but this is also for our leaders as well when they explain what we do as a team. It's important for us to think about it that way. What narrative can we build around all the measures? So by putting it together into a five-tier framework, we really want to get everyone's buy-in, our partners, our leaders buy-in in terms of um the way we um embed within the curriculum.

SPEAKER_02 18:21

Yeah, wonderful. I I really can see it coming to life with multiple stakeholders. And and it's fascinating. I I would imagine that you spent a good amount of time thinking about all of this to make sure that it would um actually work. Uh, perhaps even some pilot phases where you tested out a few aspects of it.

SPEAKER_00 18:40

Definitely, yes. Um uh the it the the framework include measures that we can implement in the future, but it is a great way to show that there is a lot more that we can do.

SPEAKER_02 18:51

Yeah, yeah. Yeah, that's great. And t take us down the road of of methodology and and then lead right into what are some of your key findings and insights from these evaluations. They're pretty robust. I I really I'm on the edge of my seat.

SPEAKER_00 19:08

Thank you. So, out of the framework, um, um apart from all the traditional measures and metrics that we put in, in 2025, we decided we really want to focus on the um impact measure on career readiness. So we were looking at um implementing a pre- and post-class measure on how much the student actually learn from what we taught within the curriculum. It is quite a simple but really a scalable approach. So in 2025, or just prior to our major delivery season, we delivered three questions that we want to um we develop three questions that we want to put into class time. Um they are around three measures um understanding, clarity, and confidence. So, to what extent are students actually understanding the content that we put in? Um, clarity, how clear are the students in terms of the next couple of steps that they need to take, and then confidence. To what extent are they confident in carrying out those steps? So we put those pre- and post-measures in most of the classes that we ran in 2025, and we managed to generate um just under 3,400 responses from students. So that um when we pair them together, pair the pre- and post-responses, we've got 995 match responses. Um, those responses cover 70 subjects that we've delivered across all of the faculties that we work with. Um, the key findings, um, I was very excited when I was doing my um pair teachers analysis, looking at uh that we actually have got um very strong statistically significant positive shifts across all the three measures. Around 75% of the students reported they have um improved on each of the three measures. And um more than that, um 25% of the students show a substantial gain. That is, on the five-point scale measure, they improve by two or more points in each of those three um measures that we've put in. So the evidence is really strong. It's really showing us that we've got um um really strong and measurable learning impact. And we uh what we put into the curriculum significantly improved the students' career readiness.

SPEAKER_02 21:26

Yeah, that's wonderful. Julianne, you must be so proud looking at I mean we met in 2017, and some of this was just like a the seed of a dream.

SPEAKER_01 21:36

Yeah, yeah, no, so proud and couldn't be more grateful for the work he has done to get us to this point as well, and having a a different perspective on you know how we reach our goals. So um, yeah, it's been wonderful working with Eva to get where we we are, but I couldn't be more prouder, and especially um, I do want to, I guess, ensure your listeners are here and that this is a team. Of 4.6 staff. And you know, this isn't 20 staff that oh, you can do anything with a large team. This is a small number of people in our team who've achieved this. So yes, incredibly proud.

SPEAKER_02 22:12

Yeah. Well, and it shows the beauty of innovative thinking to scale something that matters and to be able to stay the course and seeing that um ripple effect happen over time. And so it's it's really wonderful. And so you you have some of this data now. You you you uh it's helpful, it's on embedded career learning, it's not just on how many people came in your door for a resume critique. It's it's on the good stuff. Uh so how are you using the data with stakeholders?

SPEAKER_00 22:56

Um I guess uh the beauty with the data is it's quite simple. The three measures are easy to understand, and they help us to really, as I said, build that narrative when we are presenting to different stakeholders. So, so far, what we've done, of course, we um did share the narrative with our senior leaders. We were able to um feature what we do in the UTS employability report. We presented to the um teaching and learning community, and uh more recently um the statistics were included in the parliamentary submission as well at UTS. So uh what we're really doing, it's not just the local conversations that our consultants are having with um academic staff within the faculty, it's really um uh embedding those um information and our findings in uh our conversations with leaders as well. So those are very, very um important. And as we said, it is the narrative that we're trying to build out of the statistics that is the most important along the way, particularly in times of change.

SPEAKER_02 23:58

Yeah, that's great. And I do like how you mentioned times of change because I actually think that that's pretty constant in higher. I mean, I think you'll you're gonna get some bigger hurdles of change, which you're experiencing at large across Australia in 2025 and into this year. But um even at the beginning stages when you were starting to build this out, it there were times of change happening. And so I I think the the the seasons that you've walked this through from conceptual to implementation to measurement and now to to sharing the the the full compelling narrative of success, it's it's really brilliant. And I actually love that you have a pretty representative sample. You you're you got good data. It's it's not uh you know 37 students from one class. You're talking about across every every academic department, it's it's uh nearing a thousand data points, it's it's representative. That's that's terrific. Um so with seasons and with the future uh ahead of us. Um let's talk about the future a bit. Uh Julian, as you look into the future, what are some ways you hope to continue to strengthen your career ecosystem?

SPEAKER_01 25:18

Yeah, yeah, good question. And then I think really good positioning with all the change. And I guess something that I would like to share is something that we considered through this experience as well as being a very industry-focused university. Um we're always thinking, how can we support our employers, our industry partners, how can we help them? And I guess this approach that we've taken was our answer to that because we were seeing a shift in student behaviour as well around engagement in events and on-campus experience with employers, and we weren't getting that buy-in anymore. So it was actually if we just 100% focus on in on our students and make sure that when they meet industry, that they're as prepared as they can be, we are in fact service service in our industry. Better than if we just put them in a room one off to meet in an event. Because if we do that, if if either party can't make that event, we've serviced no one. Whereas this is more of a students are going to interact with employers in the university environment outside the university environment, and we want them to be ready for those interactions and know where they're going. Um, yet rather than just through our events. So sorry, I know that wasn't in the question, but I was just thinking, oh, we haven't actually mentioned industry, so I will yeah, we sort of have focused on students to focus on industry, if that makes sense.

SPEAKER_02 26:44

But yeah, it does make sense. It's beautiful because it now I would imagine that industry is going to say about you more and more that look at all the students at UTS that are prepared, that are ready, because they're all walking through career learning rather than uh, well, I have to dig for the good ones. And I always get some good some good talent, but I have to dig for it. No, you're saying they all walk through these learning outcomes. We're we're trying to make a system where everybody learns, and that that you're exactly right how that helps industry.

SPEAKER_01 27:21

So wonderful.

SPEAKER_02 27:22

I love that segue. Yeah.

SPEAKER_01 27:23

So yeah, sorry. Just haven't mentioned industry, we shouldn't have mentioned them. So yeah, so um, I guess as we look ahead, you know, it is doing more of what we've done, but instead of focusing on growth, really focusing on that impact. So working with our faculties and our leaders to um design career development from um in re-accreditation and course design so that it's designed from the offset and not something that we add in later. Um, continuing to add to our impact um analysis and our different outcomes as well. And then with our service model, what is really wonderful about the approach we've taken in curriculum is it frees up some time to help our students who have more higher demand needs. So um that's sort of where we're going next with it, it's over that time that's freed up. And Eva's probably thinking there's no free time, there isn't, but um just reallocating resources so that we can go okay, when we're pretty confident that through our digital services and our just-in-time offerings online, through our curriculum integration, um, then our in-person human, high human touch offerings are really focused for those who most need it. So um, yeah, and we can only do that by going through this next stage and seeing our impact and being comfortable at where we're at. So that's sort of the future for us, and we're pretty excited about it.

SPEAKER_02 28:55

Yeah. Well, Julianne and Eva, it is such a delight to have you on the podcast and to hear about what you are building. Thank you so much for being here.

SPEAKER_01 29:05

No, thanks for having us. It's been a pleasure. Thank you, Jeremy.

SPEAKER_02 29:10

And you can catch all the episodes and subscribe to the podcast and our newsletter at Career Leadership Collective.com slash podcast. As always, if you've not yet ordered the book, the Career Ecosystem Era in Higher Education, we encourage you to do so. Send us your questions, thoughts on what's working in your career ecosystem, and challenges you'd like us to address too. Podcast at Career Leadership Collective.com. We are building a new era together. Bye for now.