Episode 286: How Roblox Is Rethinking People, Systems and Workforce Strategy (With Jack Buckley)
What changes when the people function is designed not as a collection of separate HR processes, but as part of the operating system of the company?
In this episode of the Digital HR Leaders podcast, David Green is joined by Jack Buckley, Chief People and Systems Officer at Roblox, to explore how measurement, people analytics, technology and workforce strategy come together inside one of the world’s best-known technology companies.
Jack’s career spans the US Navy, academia, applied statistics, large-scale assessment and game-based hiring before leading people and systems at Roblox. That background has shaped a clear philosophy: before organisations make better people decisions, they need to be confident they are measuring the right things, with data they can trust.
Join them, as they discuss:
Why data quality and measurement should come before advanced analytics
Lessons from building a people science and analytics function during hypergrowth
Why People and Finance need a shared view of workforce data and planning
How the Roblox operating system brings people processes, technology and internal products together
When organisations should buy technology, customise it or build their own
How AI is reshaping recruiting, assessment, performance, retraining and workforce planning at Roblox
Why skills atrophy could become a significant challenge as employees rely more heavily on AI tools
How to balance rapid experimentation with rigorous evaluation and governance
Why measuring the work and skills people actually use is essential to redesigning work and creating business value
This episode is sponsored by 365Talents, a Docebo company.
Every organisation approaches skills in its own way, so 365Talents designs its talent intelligence solutions to adapt to how each business works. HR teams use them to put skills-based strategies into action more quickly and to see the impact across the whole organisation.
Find out more about 365Talents
Building or refining your skills strategy? The 2026 Skills Toolkit brings together 14 free HR templates covering every stage of the journey. It includes a business case built for your CFO, a structured framework for comparing vendors, a strategy you can take to the board, and practical tools for day-to-day HR operations and rollout.
Get free access to the 2026 Skills Toolkit
Resources:
Roblox
Insight222 People Analytics Trends 2025/26
HR Operating Model for the AI Era
Try Roblox's Coding Cookies practice assessment
This episode of the Digital HR Leaders podcast is brought to you by 365Talents.
[00:09] David Green: Over the last year, a lot of our research at Insight222 has pointed to the same conclusion: data, people analytics and a more product-led approach are becoming increasingly important to how the people function creates business value, particularly as organisations rethink work in the age of AI. So, when a mutual friend told me I had to get today's guest on the show, I was delighted because his career and current role bring many of those themes together. Jack Buckley is Chief People and Systems Officer at Roblox. His route into HR is anything but conventional. He started his career as an officer in the US Navy, became a professor of applied statistics and measurement, worked at the National Centre for Education Statistics, and later joined a startup building game-based assessments for hiring. Roblox acquired that company around five and a half years ago, and Jack went on to build its people science and analytics capability from the ground up. Today, his role combines leadership of the people function with many of the responsibilities that would traditionally sit with a CIO.
In our conversation, we explore why Jack believes measurement and data quality have to come before sophisticated analytics or technology, and the lessons he learned building a people analytics function during hypergrowth. We also get into the Roblox operating system, why the company builds or customises many of its own internal tools, and why the relationship between people and finance is so important to credibility and workforce planning. And we discuss a tension many HR leaders will recognise: how do you move quickly enough to experiment with new tools while maintaining a high bar for evaluation, governance, and evidence that they actually work? There is a lot in this conversation for people analytics leaders, Chief People Officers, and anyone thinking about how the people function needs to evolve. So, without further ado, let's get into the conversation with a brief introduction from Jack himself.
Jack, welcome to the Digital HR Leaders podcast. Looking forward to speaking with you today. You've had a fascinating career across government, education, measurement, assessment technology, and now Roblox. Can you share with listeners the journey that brought you to your current role as, as I said, Chief People and Systems Officer?
[02:40] Jack Buckley: Well, David, first, great to see you again. And thanks so much for inviting me on the show. I would like to pretend that I had it all planned out and had a perfect arc that I understood, but that's not quite the case. I started my career actually in the military. They paid for college, and in return, I owed them five years back as an officer in the United States Navy. And I think my plan had always been to be one of those academics, a professor that comes in and out of government, but probably in the national security space. When I got out of the military though, I went back to graduate school to get a PhD and I sort of fell into a department, because at the time, my wife, we had just had our first baby, and I basically went to school where she was in medical school and training. And that program had a great applied statistics subfield that I wasn't really even aware of was part of government in political science. And so, I fell into that, where I was just more fascinated with the methodology, in some ways, than even the content.
In political science, the political methodology is to political science as econometrics is to economics basically. So, I became a methodology specialist. So, I did end up going in and out of government, just not in the way I had expected. And so, I was a professor a couple of times in applied statistics and measurement. I led a federal agency in the United States that is sort of like the Census Bureau for Education called the National Centre for Education Statistics. And one of the things that that agency is responsible for is something we call the National Assessment of Educational Progress, which is a big -- of course, the US, being complex and federal, doesn't have a strong national presence in education, but we do do this giant national test to try to benchmark where kids are at different ages. And so, I got a lot of exposure to sort of very large scale problems in measurement. And again, went back to academia, then spent some time in our company here called the College Board, which is a nonprofit that does college entrance examinations and other educational large-scale assessment, some other large nonprofits. And then I sort of fell into a startup building game-based assessment for hiring selection. And actually, our biggest client at the time was McKinsey.
I came in, met the founder, who was a brilliant young woman, knew some folks on the board, sort of came in to try to help take the company to an exit. And pandemic happened, a lot of things changed very quickly. We were doing work in schools, but also in hiring selection. And we were about to sell the company actually to McKinsey and to another client in a split deal when we met folks at Roblox, and they were looking for someone to build game-based assessment to help hire our software engineers and product folks and data scientists. And we told them, "Hey, we're selling the company. Sorry, it's great to meet you, you seem like a great client". And they said, "Great, we'll buy you instead". And so, we ended up here about five-and-a-half years ago with a mandate to build game-based assessment, which we still do. But then, from there, just the role expanded to take on more and more of the people and internal products and systems functions until today. So, a little bit of accidental, a little bit of intentional. It's been a fascinating trip, though.
[06:19] David Green: You've spent much of your career working on measurement, assessment, and selection, obviously something fundamentally key to an HR function, whether it's assessing people to come to join the organisation, assessing people for promotion, assessing people for leadership, etc. How has that shaped the way that you think about people, performance, and decision-making inside organisations?
[06:45] Jack Buckley: Yeah, there's an old quote, and I won't get it right because I haven't memorised it, but it's by Josiah Stamp. The gist, you've probably heard it before, but the gist is around, the government is great at statistics and performing all sorts of analyses after they've collected all these data. But you never forget that the original figure came from some village watchman just writing down whatever he pleases in the book, before it goes up the chain and gets turned into all sorts of inference. And again, working kind of in the data collection space, especially at large scale, both nationally here and internationally, I think the very first thing I always think and worry about is data quality and measurement. And it's just sort of like, yeah, everybody wants to jump to inference. They want to reach some kind of evaluation, tell me if some program is working or if things going up or down, but often they're built on an extremely shaky foundation unless you really dig in. And I feel like that's one of those lessons that you learn the hard way, but then it's carried over to everything.
I'll sit in our executive team meetings and be looking at metrics of the business. And my head always goes immediately to, "Well, how did we really collect that? And what are they measuring? And what's the error in that? And can I really believe where this is headed?" both in the people space, but also everywhere. It's like one of those ways of thinking as a former statistics professor, that once you start that way, you can't really turn that off. And so, again, people want to talk about anything from attrition to compensation, and my head starts always with kind of data quality and measurement. Like, "Do we really believe any of these numbers that we're about to build this house on top of?"
[08:30] David Green: What would you say, what are some of the key things that people listening or predominantly working in HR should think about when it comes to data quality?
[08:37] Jack Buckley: I mean, I started off sounding a little bit pessimistic and because I always take a sceptical view, unless someone's proven to me the quality and provenance of their data. But the optimistic side, and I guess because of so many years of measuring things that are hard to measure, be it some kind of human cognition or attitudes, sentiment, I actually do believe almost anything can be measured well. And so, I think what I would certainly take to our work here, but learned through, again, perhaps until current recent changes in our federal government, at least the quality of our statistical system and their data was very good. And so, I learned, for example, like a survey, you don't just write down some survey items and throw it out there and hope that you're getting back what you get. A good survey, be it attitudinal or behavioural constructs, takes a lot of time and effort and checking through interviews and cognitive lab work and scale analysis and everything else to make sure that you're actually measuring what you think you're measuring.
That's very easy to apply to HR. And I'm really proud to say we've got a great people science and analytics team here that we built because our CEO and leadership originally were willing, even when we were small, to invest in that level of measurement. So, I can say with a lot of confidence, if I'm going to report on employee sentiment or how it may be related to some business outcomes, that actually at least we nailed the measurement. And it's absolutely possible to do that all across HR. But I do see a lot of people kind of don't always do that well, to your point.
[10:22] David Green: No, very good point. You talked about the people science and analytics team. When you joined Roblox, one of your earliest priorities was to build that team from scratch. Love to hear, what did you focus on first? What helped you establish credibility with the business, because I guess it was a new team?
[10:41] Jack Buckley: Yeah, we were small when I joined. When we were acquired, Roblox was about 900 people. We had not gone public yet on the public markets. And we were really in the middle of hypergrowth, right? I saw us double in that first year and then we doubled again. A lot of things changed. The immediate priority, of course, because our startup, which was called Imbellus, was acquired to build assessment, is I had to move all of our existing assessment to the Roblox platform. Roblox, of course, is a game platform. We don't make video games. We make the technology that lets you make video games, and it's absolutely fit. I'm a lifelong gamer, too. So, being able to work with such a fantastic team and great technology was just an amazing opportunity for us. So, we got to move our assessment and kind of rebuild it on Roblox and revalidate it and make sure that it could work to select here. But also, looking around, I went to the CEO and said, "Dave, what else do you need? You acquired our company, we've got all kinds of data scientists and I/O psychologists and cognitive folks and game design. What else can we do?" And that's where we realised together that there's not mature people analytics or people science really at all.
But given that growth that we were undergoing, I think getting a handle on recruiting data was really the first priority. I remember the very first thing, we took some of the best data scientists, who were actually building the assessment pipeline, like how you score game interactions to create some kind of valid psychometrics about your cognition, and kind of tasked them with a side work of like, "Help me build our first analytics dashboard for recruiting"; and of course, finding out that the data were a mess, that nobody had set up the system in a way that really supported good insights and observation of our recruiting. So, we really did have to start there because everything was moving so fast. From there, we just continued to expand. I think probably by the next year, we created the first employee surveys basically of anything here, so starting to observe all sorts of interesting constructs that management was very interested in, but really we had no observability. Had to build and then rebuild, more than once, our performance management system. So, working with the people leadership at the time, again, as a young startup that was just about to go public, we didn't have a formal evaluation system, which we call RoGrow here. We're actually right in the middle of our semi-annual RoGrow cycle. But that, if you've ever done that, I'm sure many of your listeners have, I've done it multiple places, it's always a lot of work. It's never the most popular thing that you can roll out, but it has to be valid and reliable. People's futures depend on good measurement there too, so we had to build that.
That probably took up most of the first year, reporting the assessment, getting a handle on recruiting, and building our surveys and our performance system. There was a lot going on in the early years.
[13:55] David Green: This episode is sponsored by 365Talents, a Docebo company. 365Talents takes a flexible, tailored approach to skills and talent management because no two businesses are the same. Their adaptive talent intelligence solutions empower HR teams to move faster, build skills-based strategies, and deliver real impact at scale. Want to learn more? Visit 365Talents.com. And don't forget to explore the complete 2026 Skills Toolkit, 14 HR templates spanning the full skills journey, from the CFO-ready business case, through structured vendor evaluation, board-ready strategy, daily HR ops, and rollout. Head over to the show notes and get your free access today.
It's interesting in the work that we do. And firstly, I will mention because my kids will kill me if I don't, they love Roblox. They're 17 and 16 now. It's been part of their life as they've been growing up, as I'm sure it is for many of our listeners with their children as well. One of the things we've observed in the work we do at Insight222, about the people analytics functions that actually create the most value, is they focus on what's most important to the organisation. You've just talked exactly there about that. Clearly, when you joined, when you're undergoing hypergrowth, you've got to get recruiting, right? It's absolutely vital to the organisation, it won't be able to grow otherwise. And then, as you're growing the organisation, you need to understand, okay, are people happy? So, then the survey piece comes in very important, and obviously connecting that to some of the performance metrics that you've got as a business. And then, the performance management piece as well. Because people flow through the organisation, you want to make sure the right people are flowing through the organisation.
What I'd like to understand as well, Jack, we've got a lot of people listening who are working in people analytics teams. Many of them will be working in smaller organisations, which are maybe experiencing hypergrowth as well. What were some of your key learnings in those years, building out the people science and analytics teams, that might be useful for some of those people that are listening and maybe looking to increase their influence and impact within their organisations?
[16:29] Jack Buckley: So, I'm going to sound like a broken record and come back to data quality again, but I have to come back there, because again, you cannot deliver insight to your organisation on a foundation of crap data, for lack of a better term. And unfortunately, the good news is for us, we were here, we joined early when we created the function when we could have input into a lot of the ways that we would set up our systems and things that we still do today, like performance management or sentiment data collection. But some of the systems pre-existed, particularly our applicant tracking system and our HRIS, our Human Resources Information System, had to have been set up. The company itself turns 20 years old this year. We were small five years ago, but the company had been around for some time. And not being able to set those systems up from scratch means you inherit a whole bunch of people's decisions who were not thinking about the downstream future of analytics. And so, still honestly, some of the things that I was very frustrated about five years ago we're only cleaning up now because you also can't stop the plane to fix the engine necessarily.
But you look around and say, "Wow, I wish I could have been here about eight months earlier and help set this system up". There's a lot of those dependencies on those early decisions and how your data infrastructure, operations people are setting this thing up thinking about hiring as fast as they can, not talking to somebody who's someday, years later, is going to have to be analysing those data and probably would have had some different decisions. So, I feel like if at all possible, especially if you're about to embark on a wave of hypergrowth, or get in there early, talk to the people who are setting up the key systems that you're later going to depend on for all of your data, be it your survey collection, your applicant tracking, your HRIS, other enterprise resource systems, anything like that. If you can't, then you're going to end up probably spending a lot of time transforming those data, trying to clean them post hoc. That's fine, you'll build a bunch of things there later too, but it's great to think about that early and try to get a seat at that table.
We're very fortunate here. You mentioned that the title here, my role, is Chief People and Systems Officer, that we have sort of the HR function, but also what would be the CIO someplace else integrated. And so, I have been able to work on those sorts of things for most of my time here, because the teams have been pretty well integrated for some years. But that's, of course, not always the case, in which case analytics is very dependent on somebody making decisions, maybe vendors, especially if the procurement people aren't really particularly technical. They're listening to the vendor and setting things up out of the box in a way that you're not involved in. So, I'm fortunate in that I'm able to bring together people science immediately at the beginning of anything filled out or any new software development.
For a small organisation, we build, it feels like, most of our internal tools, although of course that's probably not true empirically, but we do do an awful lot of internal software development for our own internal tools. That's a wonderful luxury if you can do that. Most people can't. Most people are much more tied to what's available and what they can modify, although I do think that world is changing, given a lot of things that are happening in software development around generative AI, making it a lot easier for people to build tools a lot faster. But again, if you don't have the luxury of building a lot in your own organisation, then try to get into those conversations of the buying, right, the bake-offs, the software purchases early in the setup. Don't let them outsource; don't let the IT guys outsource to some third-party consultant that the vendor brings in to help you set it up, without being there to advise on how to do it, because you're going to regret it later. I think that's lesson one, two, and three of the early days of this work.
[20:53] David Green: That's really good advice, I think. And as you said, data foundations and data quality have always been important, but you could argue that that's magnified now with AI and the expectations from CEOs on how AI can be used within the organisation as well. Those data foundations become ever more important. So, Jack, you mentioned your recent role, and we were introduced by our mutual friend Prasad Setty, who said, "You've got to get Jack on the podcast". I'm glad we've managed to do that. And earlier this year, you were elevated to the Chief People and Systems Officer role. We've got a hypothesis at Insight222 that more of the people in those roles would have spent time running people analytics in years to come. So, great to see an example of that. And as you said, your role brings together elements of the traditional CHRO and CIO agenda. So, how would you describe the role and why does that combination make sense at Roblox?
[21:57] Jack Buckley: It's funny, Roblox, obviously it's a technology company, but again, even though we grew a lot over my time here, we're still only about 3,500, 3,600 people. And of those, a good 70% are software engineers. And then, the next highest, next largest job family are technical product people, design, data science, and product managers. And so, it is a very tech-heavy culture, which means while I probably would not be someone's top choice for a much more traditional industry as their HR leader, I mean, here, if you have a special background in nuclear engineering, statistics, coming at this analytically was very welcomed by our leadership, and sort of fits here and makes sense. That might explain the Chief People side. The Systems part though I think is really interesting. So, we have what we call the Roblox Operating System, which is tooling, right? It's a lot of how employees, which we call builders here, so a lot of how builders interact with the company. But it's not just tools, it's process, it's programs, it's basically how the company runs. And because our CEO is the founder and engineer himself, he's very systems-oriented. And so, the first answer to a problem on the people side is always sort of, "What kind of system can we build around that to make sure that that's not a problem anymore, that we can control and manage that aspect of the work?"
So, the systems piece has really always been integrated, or certainly throughout my time here, more and more, kind of through this unifying theme of the Roblox Operating System. It's how do we build the company so that it runs the way we want it to, as an integrated system, and not just as a bunch of sort of one-off functions or, you know, everything from compensation to performance management to recruiting and selection, you know, is all this integrated system and tools. So, in order to kind of close the loop on this, the person who's running these other programs and processes also has to be building or selecting the tools that you need to do it. And so, it makes a lot of sense here. It's also interesting too, right? And in other industries, the CIO might be the most technical person on the leadership team, or at least probably also the Chief AI Officer these days, or something else. And here, I would argue I'm probably the least, because we've got a ton of absolutely outstanding engineers who are building foundation AI models, who are building the game engine.
[24:41] David Green: So, love to find out more about the Roblox Operating System there, which you've touched on already, and I know something I think will appeal to a lot of our listeners because, as you said, I won't say if I call it 'easier', but I'd say more companies are looking at building their own kind of technology products within the organisation for the employees within the organisation to use. And I think, again, a view is that you're probably going to have a mix of both. You're going to bring in third parties where that's sensible, you're going to build where that makes sense as well. So, what does the Roblox Operating System mean in practice? And again, you've touched on them at a high level, but what are some of the examples of the tools or products you've built for your employees, or builders, and for your HR colleagues as well?
[25:26] Jack Buckley: Sure. You know, it's interesting. When I joined the Roblox operating system, it's both an idea, but it's also a team, right? There's a team with that name that particularly works on the tooling and the processes around the tooling. And there were only three of us on that team when it started. I was basically the lead data scientist, my boss was the Chief Systems Officer, and we had an engineering senior director that we borrowed from elsewhere in the company just to help support. And so, we kind of looked at this mandate from the CEO, which was to custom-build anything we need to run the company the way we want to and not just take what's off the shelf, and tried to figure out how could we do that with limited resources. So, we started, of course, with contractors and figuring out anywhere we could beg, borrow, and steal resources. But over time, we've grown to have multiple product teams, a ton of engineers just dedicated to this work and a lot more capacity.
We started with, well, it's funny. There's sort of three, probably more, but I can think of three options usually when you're working on people tools, right? There's buying something, of course, right? So, we're all going to look and see if there's a third-party tool that's inexpensive and good enough; there's building something yourself, the other way, sometimes not inexpensive, but you may need to; but in the middle, there's usually some kind of layer of customisation, right, where I'm going to buy this thing, but I'm going to put an application engineer on it who can tailor it in some aspects to what we need, or maybe have to pay an outside third party to modify the software, something like that. We started trying to do as much as we could through that sort of middle ground. We already had Workday, for example, as HRIS. People like to complain about Workday, but you really don't want to build it from scratch if you don't have a lot of resources. And frankly, even if you do, you probably don't want to spend a lot of your time on the HRIS. But a lot of things have to touch it and people don't want to use it. And so, we ended up, for example, building our own custom layer to run performance management because we couldn't tweak Workday enough to get it to do exactly what we wanted to do. So, we had to build a native web application to run that system, but we still wanted to use all the features in the back end that are audit-compliant, and everything else you don't want to build from scratch.
So, we started looking at tools like that, right, things to help us run our performance cycle, compensation, aspects of how meetings are run. I should linger on that one for a while, because one thing that we've finally landed, I think, this year, but has been a multiyear project to try to figure out, is how we track and manage the work of the company. And I've seen us try a lot of different things under different leaders. We tried OKRs, we tried what we called internally Goal Work Metrics or GWMs. We had some older things that looked more like what we called ARKs, but they're kind of like AMOs. Every organisation has this problem of deciding what's important to work on and how to allocate their resources to it. And finding nothing that really worked, we ended up building our own both tool and process to track that work. And it is used every week in the senior leadership meeting, where we look at the company-level work tracker and it's got exactly the detail we want, right, not some other project management system. We tried those too, right? We ended up having to build that. And then, that cascades down to every level of the organisation. It's like one of those things that nobody probably woke up early in their life and said, "What I want to do someday is build this work tracking system at a technology company". But it has probably had a bigger impact than almost any tool, because it's so important to actually how we do work day-to-day.
We are taking on that same problem now on more like workforce planning. Like, how do I integrate that with what's been historically here fairly piecemeal across finance and recruiting and people, to figure out who we need when, what skills? And we're realising that no one's going to supply that for us, we're going to have to build that ourselves. So, that's one thing. There's also the more employee-facing tools and technology. So, we have part of the ROS, so the Roblox Operating System, that builders are more familiar with, just the things they work with on a daily basis, like where does this person sit; or what's for lunch today in station four instead of station three? And so, we have a bunch of more consumer apps, builder consumer apps, that just help you navigate the company, help you onboard, help you figure out where to go, what to do. Actually, we just rolled out this year for the first time a mobile application version of a bunch of those, including our one-on-one meeting tool, and we have like a meeting effectiveness, kind of rating the quality of meetings thing. So, we moved those plus the menu and then maps, and all those kinds of things onto a mobile app, so that we could help builders, just regular builders, figure out what's going on in the company.
We will probably come back to AI, because it's hard not to these days, but that was sort of one of our first tests on the internal tool side of AI software development, where we built something, we built this mobile app with a team that had never done mobile development, in about a week, and then deployed it in about four weeks; and it would have taken us probably six months a couple of years ago. And so, we're seeing that acceleration, not just on our product side, but even on the ROS, the operating system side.
[31:20] David Green: I want to take a short break from this episode to introduce the Insight222 People Analytics Program, designed for senior leaders to connect, grow, and lead in the evolving world of people analytics. The programme brings together top HR professionals with extensive experience from global companies, offering a unique platform to expand your influence, gain invaluable industry insight and tackle real-world business challenges. As a member, you'll gain access to over 40 in-person and virtual events a year, advisory sessions with seasoned practitioners, as well as insights, ideas and learning to stay up-to-date with best practices and new thinking. Every connection made brings new possibilities to elevate your impact and drive meaningful change. To learn more, head over to insight222.com/program and join our group of global leaders.
When we spoke a few weeks ago, Jack, one of the themes we discussed was how people functions can really help accelerate the business. It's not something that HR has really had the reputation for maybe, until maybe probably before the pandemic, since the pandemic increasingly so. What does that mean at Roblox and what does the people function, in your view, need to do differently for your builders?
[32:57] Jack Buckley: I think it depends a lot, of course, on what your organisation has developed in terms of its folkways and how key people things get done. I find here, probably one of my most important partners is the CFO and working with the finance team. As a people function, if you don't engage with finance as a partner, you're just going to be a cost centre that they'll come talk to you about your benefits program and how much is this costing or that costing. And so, a lot of the key planning work, of course, for any company, any organisation as a whole is in the finance side. And so, if you want people to have a voice there, you need to be working with finance all the time. Of course, we spend a lot of time, as a people function, talking to the business, talking to every level of the business. But that finance partnership is vital for having a unified voice on how to plan and really understand what the future will look like. So, everything from what's the size of the company to the skills mix, we'll talk to our engineering and product leaders about trends we're seeing, things we need to deliver on our aggressive roadmap for the company, but then we need to immediately talk to our finance partners to figure out how to actually execute that. So, I think between both recruiting and people talent management, I spend a lot of time kind of the bridge between finance and the business as a natural role here, but I suspect elsewhere too, where the people function can really have a voice.
There are a lot of other things. I mean, this is a really interesting time in technology here in the Bay Area, where we've got some of my biggest talent competitors, very large household name tech companies are in the middle of a cycle of laying off tens of thousands. But then, others that are also household names are aggressively going after everybody on their race to IPO. And so, I've never experienced such a talent market where you're both fighting to retain people who are being poached by, you know, the Anthropics and OpenAIs of the world, but also have a great market for recruiting other talent. You know, the Metas and Googles are shedding people. It's like a feast or famine where I can close a role in a few days, but then have to fight to keep somebody that was very hard to land. And so, I spend a lot of time, we all on the people side are spending an enormous amount of time talking to the business, helping them strategize around retention, not just through compensation levers, but other ways that we know, as people professionals, that we know matter a lot to why someone stays in an organisation in kind of coaching more junior managers how to have those conversations, and figure out what really motivates people, in a way that in other years has felt more academic, but now feels like a daily struggle. Fascinating time just on the people side.
[36:01] David Green: We're going to look at three more topics, Jack, strategy, governance, and experimentation; and then, skills and capabilities in the people function as we look forward. So, let's start with people strategy. Again, when we spoke a few weeks ago, you shared that you've recently been in the process of redesigning Roblox's people strategy. What were the biggest priorities that you wanted to address across, I think you called it the five Rs: recruiting, reward, retention, retraining, and restructuring?
[36:32] Jack Buckley: Yeah. So, in stepping into this job in the middle of the AI, I'm going to call it a bubble, but the AI explosion, meant that a lot of strategy there was a little bit outside my control. I had to come up with a strategy to support what I think of as AI acceleration of our work, and I'm not talking about just the people and systems function, although we do care a lot about that too, but how to support Roblox overall, how to support our builders, again, who I said are predominantly engineers, in AI acceleration. And so, that actually has cut across all those Rs. In recruiting, for example, we've had to redo job analyses and figure out what are the effective practices that engineers use working with generative coding tools. We're seeing a lot of spend there, obviously. The cost is getting real, and so, we want to both select people and if necessary, retrain people onto how to be more effective in their engineering practices around these new technologies. So, we have been studying what makes folks effective internally on the use of generative AI and code development, and then building that into our selection processes where we can. So, we're working on some new assessments of tech screens that would help detect who's good at prompting coding engines, as opposed to old-time traditional coding assessments, you know, and then other ways that people select engineers.
A big part of that too for our early career hiring in the recruit space has been developing, rolling out a new game-based assessment task to measure computational thinking. So, our last cycle, our game-based assessments measured systems thinking and creative problem solving, which we still care a lot about. But we are trying to move away from these more traditional coding tests to a way to measure computational thinking as a more abstract thing that we know that you need, especially if you're going to use generative AI to make a lot of your code. I want to know that you're using it well, not at the cognitive bubble. So, we've been pilot-testing this new assessment task, which we're going to roll out this summer with our new early career cohort. It is basically, you have to direct a robot that's building habitats on Mars. Under the hood, of course, we're measuring the way that you use key concepts in computation, that you don't just kind of brute-force this thing, that you're looking for elegant ways to save time and optimise your code for the robot. We will have that, we'll have a practice test available for the public that sort of shows off that block code and lets people practise pretty soonish. But that's been a big effort here trying to sort of get our game-based assessments ready for an AI-accelerated future.
On the reward side, again, I mentioned competing with these still privately held but threatening massive IPO AI companies for some of our top engineering talent. We've had to come up with a lot of clever ways to compensate people, both to attract them, but also to retain them there. And so, our compensation team has been working on, for example, milestone-based compensation, where I'm not just trying to match an offer by giving you tens of millions of dollars, but I'm trying to tie it to key deliverables in your domain of AI, which again is very common in other fields, but not so much in tech. Also, we had to change the performance system, figure out what key competencies are necessary for success in an AI-accelerated coding world, and make sure that that's part of measurement for promotions and compensation. There's a lot of interesting -- we're fortunate to have good partners and colleagues in our engineering acceleration team on the engineering side. And so, together, we've had to both roll out new tools safely for people to use, like Claude Cowork or other internally-built AI tools, but also monitor use and observability. So, I wasn't expecting on the system side, probably if you'd asked me a year, year-and-a-half ago, would we be spending so much time on just tracking what people are doing in these tools. But we are spending a lot of time on that, again, partly because the cost is real.
I mentioned the R of retraining. We are finding out, of course, even as a tech company based in San Mateo, California, there are still a lot of builders that are not actually comfortable with these new tools, and they're sometimes afraid to tell us that. And so, we have rolled out, for example, when we had a safe internal launch of Claude Cowork to the whole company and not just tech folks, I had over a third of the company take a voluntary training module that we had built, and just kind of put it out there with minimal advertising. Like, "Oh, if you're not really sure how this works..." and it was like 1,100 people signed up immediately and took this. So, there's an appetite for reskilling, that classic just-in-time adult learning problem.
[41:59] David Green: I know that you've also been thinking a lot about, obviously, things that support all those areas, like governance and creating safe environments for experimentation. What have you learned about creating an environment where teams can move quickly, but still make thoughtful and responsible decisions?
[42:17] Jack Buckley: I mean, that is always a tension. It's a tension everywhere I've ever been, other industries and government, but it is a particular tension, I think, here right now, partly because of, given the nature of our workforce and where we are, there are a lot more people who are extremely excited about this technology than they are fearful of it, which may be the opposite elsewhere, which means they're clamouring for access to things and often well ahead of governance, well ahead of legal, of safety, of IT security, things that I know are going to be a problem. And so, it's quite a balance to be part of the sort of AI governance folk, committee that's saying no to a lot of things, but still trying to enable people and coming up with ways to push where we can.
I think the piece that maybe is underappreciated, because every company is dealing with that problem, but the part that for us is really key to our DNA is also evaluating the quality of these things, so well before the generative AI explosion, partly because of when we created this function, I'd spent a lot of my time also as an evaluator doing a lot of contract work, evaluation of large-scale federal programs, philanthropy, so empirical evaluation, and a lot of my folks have similar background. I don't want to turn on a tool until we've demonstrated that actually it adds value, whether or not we built it or we're buying it third party. And so, we were already always like that. And my predecessor in this role had a similar mindset. And so, recruiting doesn't get to go buy a thing and turn it on. We have to evaluate it and see if it's worth it first. So, a lot of pilots, a lot of field tests, a lot of I/O psychologists and people scientists involved in the selection and deployment of tools. I've tried to hold that line in this new era, like we don't just use some generative AI skill that you've written to write your performance evaluations. I need to know that you're not doing something bad with that.
So, being sort of, unfortunately, the no voice often on not just safety and legal issues or security, but also on sort of rigorous evaluation of the outputs of these things, I'm convinced that's the right position, but it is sometimes a very lonely one, when everybody in design wants this new thing turned on right now. And so, I think both from the people side and the system side and the IT side, I'm pushing to resolve those questions as fast as you can. But as you know, sometimes you only get the data as fast as the people do the work or do the thinking. I can't accelerate everything. And sometimes, it's going to take a month to get enough data to answer these questions. And so, I like to say we've been balancing this well. I'm sure if you ask some of my peers here, they might say we are too slow, because you're always holding this quality bar for implementing these things and just go faster. But I feel like it's part of how I was trained and raised, is you don't turn something on if you don't know what it does and how well it works.
[45:42] David Green: I'm really interested in your answer to this, because obviously you're only recently into the people function, the HR function. How do you think skills and capabilities needed in an HR and people function are changing? And again, I won't hold you to this, so it's a hypothesis, so your hypothesis, what's your hypothesis on what will separate the highest-impact people leaders over the next few years?
[46:07] Jack Buckley: I've talked about AI a lot. I'm quite a sceptic in a lot of ways compared to peers. To me, generative AI, transformer-based generative AI is absolutely going to replace some things like other methods of natural language processing, very narrow things that we used to use other algorithms to do, and you'd be crazy to do that now. And that's going to have sweeping change, but in a really narrow area. What you're seeing is people applying a pretty narrow technology and pretending that it's something like general intelligence and can do all these other things. I think the people functions that are successful are going to very quickly figure out that that second part is mostly fluff. We are going to end up in a wave of automation, and automation that we probably could have done before; we just didn't have the will to do it because somebody wasn't telling us that AI would do this for us. I think the successful functions are going to get the most out of that automation, limit the AI work to where it actually adds value. But when they're done, they'll realise that a lot of the work they were doing was not effective or not a good use of their time. We'll be successfully automated and it'll free them up to really do what people are best at.
So, I think navigating through the fluff and the vendor hype and getting to what you can really automate that you didn't need this new technology for, but maybe you had just been ignoring and throwing people at for a long time, is going to be a big dividing line for who's successful and who's not. That's my best guess right now, just from looking at a lot of people and vendors and our internal tools, like what are we deploying that's actually useful versus it seemed like a good idea, but actually it was just probably silly.
[47:51] David Green: I suppose, ultimately, the most successful people leaders will be what they were before AI actually. They connect what we do in HR to the business. And if they don't connect it to the business priorities or the employee priorities, then they can only be so successful or probably not successful at all. So, we've got to not just look inward. And there is a tendency, has been a tendency over the years for people functions to do that. And actually, what that does, I think, Jack, it connects quite nicely to the final question, which is a question we're asking everyone on this series of the podcast, and maybe this is an opportunity for HR and people functions. How should HR redesign work to unlock business value?
[48:40] Jack Buckley: Well, so, probably not surprising to you, I'm going to come back to measurement. It's trite, it's a truism that you can't manage or improve what you're not measuring, but it's true. I think the interesting issue here, like, it's easy to measure the wrong thing. I think what's going to help our work really unlock business value is measuring the harder to measure things, measuring the right things, what skills are the employees really using on the job? What does their output look like? What are they doing? And then, we can improve it and then drive, both through selection, through retention, training, we can actually improve the human capital of our organisations, but only if we built that measurement foundation first.
[49:22] David Green: Well Jack, thanks so much for being guest on the show. It's been wonderful to learn more about the work you and the team are doing at Roblox and the Roblox Operating System. Absolutely fantastic. Where can people connect with you and find out more about the work that you and the team are doing at Roblox?
[49:40] Jack Buckley: Probably the first place, easiest place is our website at about.roblox.com, which is kind of a general overview of Roblox, but people are always welcome to reach out to me directly too.
[49:50] David Green: Jack, thanks very much for being a guest on the show. Hope to see you in person maybe at an event, maybe in San Francisco at some point in the future soon.
[50:00] Jack Buckley: Wonderful to be here. Thanks for making the time to talk.
[50:04] David Green: Jack, thank you so much. It was great to finally get you on the show. And I really enjoyed hearing how your background in measurement has shaped everything from people analytics and selection to workforce planning, technology, and the wider people strategy at Roblox. One point that particularly stayed with me is the importance of resisting the temptation to confuse access to more technology with better decision-making. As Jack put it during our discussion, getting an answer to almost any question is becoming easier. The challenge is making sure it is the right answer.
So, turning to our listeners, I'd love to hear your perspective. When a new AI tool comes into your organisation, what evidence do you need before you are comfortable putting it into the hands of employees? Come and find my post on this episode on LinkedIn and let me know in the comments. If you'd like to learn more about Roblox, you'll find a link to their website in the show notes. And Jack is also very open to people reaching out to him directly. And if you enjoyed the episode, please share it with a colleague who would find it useful, and subscribe so you don't miss what's coming next. One last thing before we go. If you'd like to keep up with what we're working on at Insight222, follow us on LinkedIn, or head to insight22.com. You can also sign up there for our bi-weekly newsletter for the latest thinking on HR, people analytics, and workforce transformation. Right, that's us for today. Thanks for listening, and we'll be back next week with another episode of the Digital HR Leaders podcast. Until then, take care and stay well.