Ronan Murphy
Good morning, everybody. Thank you very much for coming down into the bowels of the convention center here at the AI Expo. Good to see so many people love a bit of tech policy in the morning, and you’ve, you know, you’re lucky, you’ve got what I think is definitely the best panel at this event today, at least, and I’m going to make some very quick introductions, and then we’re going to start talking about what we came here to talk about, which is AI adoption, how it’s happening here, how it’s happening in Europe, how both sides might be able to work together a bit better, and what the future holds in that regard. So, I’m delighted to be joined by the Deputy Chief of Mission for the Delegation of the European Union to United States, Ruth Bajada. Okay. Good. Thank you very much. We’re off to off to a flyer. Zach, Zach Whitman joins us, who is both Chief Data Scientist and Chief AI Officer at the General Service Administration here in Washington, DC. Daniela Braga is the founder of Defined.ai CEO, based in this part of the world, but with offices on both sides of the Atlantic. That’s something we’ll hopefully delve into a little later. And Glenn, Glenn Parham has joined us from Meta, who is public policy manager there, and he’s a friend of CEPA, and we’ve had him at several of our events in the past. So, thanks again, Glenn, for coming, and I’m going to start with you, Ruth, if that’s okay, and maybe talk about some of the very recent developments. Actually, last night there was a simplification announcement that impacts the EU AI Act, and I bring that up because Europe might be considered a little bit behind on aspects of AI. The big, big firms are generally American, some are Chinese, but adoption represents a real opportunity for the continent, for everybody. And how does that, how’s the recent developments impact that? And what are you doing to encourage AI adoption in general?
Ruth Bajada
Thanks a lot. First of all, I always come here to the convention center and discover a new part every time I come here. So, thanks a lot.
Ronan Murphy
Thank you for coming.
Ruth Bajada
For inviting me this morning. I see some friendly faces in the crowd as well. I’m not the AI expert, the real AI expert are sitting in the crowd, so any detailed questions would go to Denise and Guillaume afterwards. But what the EU is doing? You mentioned AI adoption, that is it’s true, EU has been slow and numbers have shown it, but we’re catching up. I think we started off on AI adoption across the continent at 80% last year. We’re at 91% this year. We’re doing some good work, also in terms of, as you mentioned, listening to companies and listening to experts, we do love to legislate, and, and we take pride in that, but we also take pride in knowing how to listen and adjust what what needs to be adjusted. So, last night, actually, together with the European Parliament, we agreed on a simplification procedure, which will allow more time in the adoption of the AI Act, and there are going to be a number of, you’ll see a number of new dates coming up for adoption by 2027 and later, and of course, a piece that is close to our heart, that when it comes to looking after minors and looking after children across across our continent, and we also find there a lot of, a lot of support also across the states in the United States when it comes to protection of minors online, so part of the simplification will be about that. We’ve also just announced most recently a way you know we’re we’re a big market in Europe of 27 member states but we have we also hear companies when they come to invest in europe and they find different 27 different legislations and 27 different labor laws and, what we have done is now we have announced what we call the 28 regime, that’s EU Inc., and that simplifies the way for a company, so even as a small and medium-sized enterprise, or a company wants to invest in Europe, they don’t have to deal, and they want to invest across different member states, they don’t have to deal with 27 different legislations, but they can adopt one that’s the EU framework, and that’s one regularity framework where labor laws will, you know, they will have to adopt one form of set set of rules rather than going through the 27. So we hope that with that, next time we meet, we will be at maybe 99%.
Ronan Murphy
Okay, that’s something to look forward to. We’re going from what is comparatively high level, we’re looking down, and there’s regulation from from from top down in Europe, particularly when it comes to AI. There are plans for adoption, there’s the EU AI factories, and so on, but I’m going to maybe go to you next, Zach, and talk a bit more about the practicalities. Here, you’re at the coalface. You’ve been rolling out the US AI. You might tell us a bit more about that, what it is, and what are you finding in real terms, day-to-day terms, are the challenges in encouraging adoption and making it happen? Because it’s easy to talk about, it’s easy to put it in another presentation, you know, you can throw it up there, but then there’s the reality.
Zach Whitman
Nothing’s better than putting AI in a presentation.
Ronan Murphy
Yeah, solves everything.
Zach Whitman
Don’t even have to do any work. USAi is a cool project that we’ve kicked off in response to a broader initiative that we saw from the White House in leading AI adoption across the federal government, the main focus was to do it in a transparent and trustworthy way. The main concern that we had going into this was when it comes to federally adopting AI practices was how were the AI models being employed, in what use case were they being used, and what decisions could come from those things, and so we had to go through and enumerate all the different use cases that AI had in our general ecosystem. With the USAi type construct what we ended up thinking about was, if we have all these AI use cases occurring across these large federal enterprises, how can we realistically and technically keep track of these specific uses of AI, it’s really easy to say, like, yeah, sure, the policy is that you have to let everyone know exactly how you know AI is being used in this process, but as a policy control, it’s lacking the technical effect that you would want from an audit perspective. I want to be able to see all the log data to say, you know, these are how these are the prompts, responses that are occurring in our current ecosystem. This is how it’s being deployed with what organization, with what office, and for what outcome. The big fear would be that you would use AI to make procurement decisions or rights or safety impacting decisions without proper oversight or transparency. So, what we did at GSA was we built an infrastructure that allowed us to connect to all the leading foundation models in a central place, so a one-stop shop for your chatbot type solutions, which were a basic necessity, and we were seeing a lot of internal pressure from employees in terms of I would like to have some sort of tooling that I can use for internal work, and I don’t want to rely on public tools for non-sensitive use cases, so we were able to establish a general chatbot for that initial impulse of user demand that we saw, but then secondarily we’re able to consolidate all the APIs, so that all these subsystems or subprocesses that were either machine to machine or incorporated into some other system that you already were using as your workflow could inherit those models, and effectively we’re able to hydrate all of these use cases through a single pipe that gave us full transparency into what was happening in our organization, but also gave our users the ability to easily pick between approved models. Nothing is more encumbering than wanting to build some sort of process, and having to go and open up 16 tickets to get access to the three different hyperscalers and all the different modes with which you would want to consume those models. So our focus was really on making life better for the user, our employee base, so that they can consume models, the leading models, as quickly and efficiently as possible, while also providing the telemetry and the control surfaces for our CIOs and CISOs to effectuate policy, so that certain models could get in and certain models were restricted, and what we’ve seen is that that we saw a tremendous amount of organic demand from the other agencies, so they saw what we did, and they’re like, could we in some way use that, is there a way we could get in onto this platform, and so what we’ve done is we’ve turned the internal platform, which we built originally called GSAi, into this USAi pilot program, and the idea there is that we wanted to see if there was like a real system demand across the federal landscape for a platform like this one, in which we could offload or centralize a lot of the evals that we do for safety and performance of the different models, so we can contextualize the behaviors of these different systems and allow for the right model to be used for the right use case or sub component of specific use cases, and then also provide a general platform for these other agencies to empower their workforce with the same baseline capability and centralize the access of approved AI models. It also provides the ability for you to host your own model, so the idea that you would develop your own internal model and then turn it back to your workforce is something that we’re actively building as well. Functionally, we’re thinking very much in the sense of from an employee, how can we give them access to these latest and greatest tools, without having vendor lock and control the stack and interface directly with our data. This separation of concerns gives us a lot more flexibility, as this space is too dynamic to really lock in on a single vendor. So, we’ve seen a lot of success so far in terms of general adoption, and I think it’s born out of the practicality of the on the ground true. Within the use cases that we’re seeing, but we really don’t know how far this is going to go. We see USAi as a potential to continue and to continue to support and grow, but fundamentally we’re in a moment of just dynamic experimentation, and we’re really excited to see how that’s been kind of like picking up in steam.
Ronan Murphy
So dynamic experimentation, that’s not something usually associated with government work.
Zach Whitman
I don’t know if they love it, but the funny thing is, we’ve seen just a tremendous amount of interest in taking a risk-based approach rather than a very policy-driven compliance approach, and so that risk-based approach has allowed us to build a risk portfolio of adoption, rather than going through what would typically be a hyper conservative environment. We don’t have time to wait and see how this goes. Our employees demand it, the public demands that we are more effective, and we have efficiency drivers that we need to make sure that we’re, we’re meeting, and so functionally it’s just it’s a, it’s a scenario where the better approach of a risk-based approach allows it affords us to be a little bit more progressive in our deployment of these new technologies,
Ronan Murphy
And are you, are you seeing any results of actual results within, within government departments, within units?
Zach Whitman
Yeah, yeah, so the main, the main issue that we’re facing is, how do we, how do we calculate ROI, right? Like, what is the value of a chat bot right now? The pricing right now is on the market is quite dynamic, and the question is, like, if we deploy a chatbot to your employee base, what is the true ROI there compared to the token consumption? What is that worth to the organization versus an API that you connect to your agent decoding or to an agent swarm or subsystem, where you’re seeing a lot more dynamic consumption. We are actively working in close collaboration with a lot of our tenant partners about how we can measure the effectiveness and the utility of these tools in their deployment of the use case, to say we’re seeing time efficiency savings, we’re seeing better service delivery, we’re seeing, you know, fewer contracts being written because we’re able to write code. Those types of questions are very open, and we’re working hard to understand, like, what is the true value of this product and service. We know that there is something there, but to quantify it in a reliable way, yeah, consistency is really the main goal for us right now.
Ronan Murphy
Well, with a lot of what we’re talking about, it’s a bit early to tell, and I think ROI is something dynamic you must hear about all the time from your customers, and internally you’re looking for a head. What’s the point? Why would I go and buy Defined.ai’s products? And the same Glenn’s nodding his head, he’s wondering about the same thing. So maybe, maybe give a very quick introduction to what Defined.ai does. Zach’s got two job titles, one of them is Data Science Officer, and I think that leads in the data is a very, very important part of what you’re doing. Maybe tell us a bit more about being a European, is doing business on both sides of the Atlantic in an AI company.
Daniela Braga
Yeah, then so Define.ai is an 11 year old later stage scale up. I am originally Portuguese, and I started in the AI world 26 years ago in Europe, which also shows the talent pool and the capability of the European universities to create AI capable talent way before it was hype, as it is right now. There’s a lot of, there’s a lot of debate around the US and Europe, and the talent, and things like that. I think I can tell I am the living example that I’m an European, graduated with a PhD in Europe in AI before it was called AI in 2008 and I leave corporate America with Microsoft, which found me there during my academic career, and with seven years at Microsoft, moving around the world, I landed in Seattle eventually, where I built this company, and this company is the third, what I call the third pillar of AI. You have models, you have compute, and you have data, and you could argue that you also have the talent part. Of course, we always need people to build innovation and technology, but the data pillar was what always captured my attention throughout my career. You cannot, especially when you move from a paradigm of rules when you were teaching computers how cognition and human computer interaction worked through rules to data driven patterns, which now with our compute power we basically just dump terabytes and terabytes of data there, and we hope that our compute and the massive frontier models will make some sense of it. It became much more, much more difficult to understand what’s going on there, because it’s all about the quality of the data that you put in. So, yeah, so explaining that I started my career in Europe, I built. I started the company in Seattle, and, but I immediately set up an R&D center in Lisbon, Portugal, which is still our largest office. In, I mean, we have several. We have, we still have headquartered in Seattle. We have Seattle. I’m here now for the last three years, and we have critical mass in the East Coast as well, but we mostly West Coast now, East Coast, and Lisbon, and Portugal, and Spain. It’s been always the case. We’ve been always there, and now in Riyadh as well. So it kind of gives you the, I mean, and on ROI question, of course. I mean, we work with a large AI world labs in the world, which are, I mean, which originally, when I started the company, we were, they were, maybe they were the big tech, of course, the usual Microsoft’s and Google’s and Metas, and which was Facebook back then, and Amazon, and Apple, and IBM, and then suddenly, and of course, those are those those guys have been always there, they had been leading the world innovation in AI, I mean, I come from one of them, of course, Microsoft, but but what I’ve seen is the things changing from, I mean, telco companies were leading the way, even in Europe, a lot in terms of AI research and development back then, and suddenly you have what we call the AI digital natives, which is the OpenAIs and the Anthropics and the Reflection AIs and Mistrals and all those guys who are not native in a are born in AI and became as big as any other clients of ours. Our clients, because they measure AI is on this race they measure results all the time, and the results are always, and these ROI is measured through data as well. So, you have to have pre-training data at scale to train their models, but you also have to have the old set of the data test set to benchmark every build you’re doing, so this is absolutely critical, and this is what we help our clients do through closed source copyright clear ethical source data.
Ronan Murphy
Thank you very much, Daniela. And turn to you, Glenn, you’re with one of the big, one of the big players in this, in this field, and you’re, you’re trying to encourage adoption in different ways, because it’s your businesses, you have your users, you have your customers, your users, and what’s the, what’s the, what’s the watchword? Why, how do you go about promoting adoption of the products that you’re now offering, and what, what are you finding that’s challenging, and then global company as well, that’s also very important?
Glenn Parham
Yeah, absolutely. So, my background is in national security, spent the past years in the Pentagon, being the technical lead for generative AI. Hopped over to Meta about a year ago, where I now work on national security AI adoption. So, taking our generative AI models, it’s called the Llama family, the herd, they refer to it as, and bringing it to our national security community, as well as some of our allies. So, it’s been incredible to be on this side of the equation now to see how people are really like using our models all over the place. I mean, I would say really the reason why I jumped over to Meta, frankly, is because of the emphasis on open source and open weight models, especially in the national security context, where you’re dealing with air gap networks and really constrained cloud environments, and so forth, classified networks, you know, you don’t, it can be difficult to bring online AI models from a lot of the, you know, some of our incredible competitors, and so what I saw firsthand in the Pentagon across the DoD was the dominance, frankly, of Meta’s models, our Llama models, since, you know, ChatGPT came online up until today, and you know what we’ve seen over the past, you know, year, I would say is an emphasis and expansion to a lot of our European and NATO allies who are also in the defense context really extensively using our models, especially because other model providers aren’t necessarily readily available there, and so I would say by and large our adoption strategy has been that emphasis on open source, we do a lot of work partnering, talking directly to the government, partnering with almost every defense company you can think of, because we know that’s where the real innovation is, and because we’re open source, they literally just go to our website, pull it into their own environments, and are able to get to work, and so we’ve just done a bunch of proof of concepts, we have last year we actually sent Llama to the International Space Station, we call it Space Llama, just to demonstrate how we could have our models running in, you know, crazy sorts of environments that are disconnected from the rest of the world, so you know our adoption strategy is I would say the first marker and expansion was back in 2024. We amended our licensing for Llama to allow and essentially authorize the US government, as well as some of our allies, to use Llama for national security contexts. We spent a year after that partnering with a lot of the main defense companies just to proliferate our models, and then it was awesome. A couple of weeks ago, we were able to sponsor our first national security AI hackathon right here in the, in the DC area, and that was incredible. We got to see how our models across the board were applied in all sorts of different domains and contexts, and see how people were able to build around it, what they’re able to deliver, and a lot of good things and projects are coming out of that specifically. So I think you know it’s, it’s, I really do like this job, just because it’s frankly probably easier than a lot of my counterparts at other AI labs, just because it’s open source and you know people can pull it into their own environments overnight and get to work.
Ronan Murphy
So the fact that you come from a national security background and you this is what you’re working on, and the open source models are used in there in defense and so on, and kind of leads into a question that hovers over all this, which is the question of sovereignty, whether it comes to tech or it comes to AI specifically, and clearly you’ve taken a position at Meta. It’s it’s open source, open-weight model. This is the way it’s going to be, where anyone can use it. You can store it wherever you want, you can do whatever you want with it. Is that it? That’s the there is no water’s edge, like there is no who can use it, who can’t use it. You don’t, you don’t make that distinction.
Glenn Parham
So we, we only make our models available to to the US and our allies and our licensing and so forth. You know, who knows? Yeah, we’ll just leave it at that. But back to your point on the idea of sovereignty, we’ve seen such an investment in Europe in our models and leveraging our models, because it’s open source, because they can pull it into whatever cloud or on-prem system that they have configured, and get it spun up, and use it, and not just, you know, run inference of the model, but do tests and evaluation of the model, right, fine tune it on their own sensitive information with the assurance that none of that data or information is going to egress back to other, you know, back to other countries and be used to train on their data for, you know, other purposes. I think it’s the assurances that open source brings that really makes it attractive to use open source.
Ronan Murphy
Yeah, so I mean, you’re right, it is. It is two distinct questions. There’s, you know, restricting those who gain access, and then there’s building something sovereign that you can use yourself and no one else can access, but you’ve made that decision as a user, as a company, as or as a nation state, or as possibly as the EU, right? I mean, sovereignty is a big topic. Independence, tech independence – there are different versions of it. At CEPA, the Center for European Policy Analysis, for those who don’t know in the room, we are very much in favor of transatlantic cooperation. We see it as a fundamental relationship for both sides. We actively encourage that cooperation. We want to see more of it. The sovereignty perhaps challenge the notion of using things brought in by other people, or is there enough scope in the way Europe is approaching sovereignty, you think to always be able to use the latest and best from your trusted partners. What can you tell
Ruth Bajada
We thank CEPA for always putting the transatlantic relationship to the fore, and I can assure everyone in the room that also the European Union wants to see this relationship grow further. I mean, you know, apologies, I forgot, I forgot your name, but you just mentioned that you have a lot of customers in Europe. Let us, let us start – yes, we want sovereignty, and yes, we want to ensure that the protection of our European citizens, but let us not forget that American companies are thriving in Europe. Yes, President Trump complains about that, that you know, more American goods come to the EU vis-a-vis American goods being sold in Europe. It’s the complete opposite in services. American companies today, if it’s Microsoft, Microsoft makes 40% of its global profit in Europe. Google has more users in Europe that it than it has in, and then it has in, in, in the United States. So, and you know, we want this, and we want this to thrive, but we do regulate how, how we would like companies to behave in Europe to protect the European customers, and again, there we have also heard also companies that have been invested in Europe, and European companies that want to grow in Europe, and that’s why we have gone through the tech legislation, and we’ve come up with this package of looking at the legislation and seeing ways how improve, how to improve that, so that’s you know, that’s what we will see, that’s that’s what you will see, and you will see also a lot more investments in AI across the continent, and starting from, you know, the most advanced chips in the in the trade deal that we signed in Turbo, in Turnberry, with President Trump, there is also the amount of 40 billion chips that we will be buying from the United States, and that’s also that also will be used to to make and to strengthen the European continent as an AI continent. We already have 19 factories, AI factories across the continent. We intend to build more. We have, we have also some of the best supercomputers in the world. They were, they were initially set up for scientific purposes. We are now strengthening those with further GPUs to use them for, for AI. So, you know that we have a plan, and that plan also includes a transatlantic plan, working also with many of the American companies that want to invest in Europe, and you mentioned also, incidentally, one area where we see huge growth is defense and tech, as the European continent now is really, and the EU is really investing in its defense, something that we have, we should have done earlier, but now it’s an opportunity, and there we see huge interest from American venture capitalists that not only want to bring European companies here but also want to invest in Europe in in defense, defense and tech and defense, so you know there are huge possibilities on on both sides. Yes, we speak of sovereignty, but we also know that it’s by working together with the United States and with also with other countries, we have trade and technology council with with India, where we have a strong conversation on on tech with them, we also work with Japan, South Korea, and a number of other countries that are like-minded, where, where we see a lot of potential to work together.
Ronan Murphy
So, sovereignty doesn’t mean that it can’t, it has to be European. So, the Buy European plan for, for tech won’t, it won’t, it won’t, won’t preclude using
Ruth Bajada
Again by European in many of, so when we have money for European, when we have, when we are using European budget to scale up and to support the increase of European companies, and that’s the part of the European budget, it there will always be, of course, because it comes from European taxpayers’ money. It’s obvious that there will be, there will be a European preference within that. That said, all legislations that you will see coming out on that, there we will always be living up to our international obligations. So, even when it comes to procurement, if we had made deals with the United States that on certain areas in terms of procurement we will be working together, we will honor all those areas that that are listed in in government procurement agreements, for instance, so you know we, you know, we see the big, you know, people get stuck sometimes at the big word, but then do not read what, what comes, what comes after it, and the potential to work together. But yes, it is our responsibility to make sure that European companies thrive and grow, and, and you know, and potentially come and grow also in the United States.
Ronan Murphy
Can I ask you a direct question? Daniela, does the sovereignty come up when you’re selling the clients say, “Oh, I need it to be in a certain way or a certain place” or is it “I just want the best system and I trust the company that’s offering it?”
Daniela Braga
In general?
Ronan Murphy
In general.
Daniela Braga
Models, I guess there’s two types of clients, there’s the enterprise clients who care about their brand reputation.
Ronan Murphy
Yeah.
Daniela Braga
And they definitely care about how the perception of and the interactions with the governments they’re in, so even though I mean I have to say even though some of the vision models, the best ones are coming from China, we are, we should be all using the best models, depending on the on the application, but we can’t, right, and actually we can’t, because we have a different, this was not the case five years ago or 10 years ago.
Ronan Murphy
Okay.
Daniela Braga
Leaderboards from Hugging Face are there and show that when and other models are tough, so we cannot ignore that part. But then we have all the part of how were they built? We come in with the how were these models built, because we know we know, and we can tell, and we work with everyone, so we know how these models are built. We can reverse engineer how these models are built, and it comes down to – was the data acquired, how was the data sourced? I mean, and here there’s no one that is currently, everybody in the East and the West is crawling the web, everybody, everybody’s crawling the web, so there’s really no, which means copyright infringement, which means a whole whole impact on content creators’ economy and livelihood. So then, where is the difference? Where does light, where does the difference lie? And a lot of companies are doing a great job making it transparent, like IBM, like Meta, putting as open source models, because the closed source models are not going to make it open, obviously. That being said, they are enterprise clients, so they work with enterprises, they know that their clients care what’s in there, they don’t want to have a copyright legal lawsuit around when they test, when they use it internally, or when they integrate it, so we know that if these models are showing that we pay for this data, we use this data, this is even though it’s closed source data, I think that makes a huge difference, but then there’s a lot of people, and the consumer, because they’re obviously consumers, cannot and should not be able, as they cannot afford to pay for a model that is closed source, or that they know that is there’s, they cannot afford, so what are they going to do, they’re going to use the lowest hanging fruit that is available, despite what’s in there, and probably even being okay with this model taking their data away. So, I guess that’s the two differences: there’s consumers, and there’s the enterprises, and there’s a brand reputation, and where you want to play as a client.
Ronan Murphy
Yeah, of course. So, there’s level of understanding, and then there’s also your appetite for risk, and there’s your available budget, all factors that must come into play for you as well Zach, do you have it’s America first, is that the, I mean, it happens that that’s where the tech firms, the big companies are, so maybe it doesn’t just doesn’t arise for you, but what if some German firm came with an offering, are they ruled out or they ruled in, or just give them a good look.
Zach Whitman
So, like, yeah. And right now, a lot of the platforms are the one gov deals that GSA are leading are primarily meant to spur, or are in furtherance of spurring AI innovation within America. Right?
Ronan Murphy
Yeah, it’s a procurement question. Yeah.
Zach Whitman
The direct relationship that we are trying to execute on, and I, but I think the point about, like, where did this come from, how was it generated, running evaluations and trying to understand fitness for purpose, fitness for use, and you know, fundamentally, what is the sustainability pathway of these models, is a core concern for all of all of us involved with the space. So, like being able to understand the leaderboards and looking at certain models, maybe outperforming, or, you know, you know, beating certain evals to us in our platform and on our approach generally is kind of fundamental to the offering. We don’t want folks to go in blindly and pick a point in time assessment, because at that point when they checked it, it was the top of the leaderboard. We believe much more in distributions, continual evals, building our own evals for specific. Use cases, and making sure that agencies are making the most, the most informed decision possible, which will oftentimes break the heuristic of always going with the bigger, latest model. We’re a big believer in the open source models for tailored applications, and in certain circumstances, you can improve the models to get to the outcome that you’re looking for, as long as or provided that you have established a core set of evals that you care about related to that use case, and so we’ve been really big on not only empowering like the main constructs in the main AI labs that we are required to, but on the other side we really want to make sure that the decision point as to why you would choose specific models, regardless of what Guard in Europe potentially provided, is the most informed and most data-driven based on your own internal subject matter expertise, and so we’ve been really focused on not only providing like leading AI models, but also providing the open source model alternatives, smaller models, specific models for very specific use cases, and then empowering and allowing for agencies to take their subject matter expertise, bring that to bear to develop their own custom evals, and then ultimately provide a more accurate or precise solution, and also giving them the platform to regularly re-evaluate over time to assess for things like model drift, and so our main, our main focus, yes, is in prioritizing the AI, American companies, American AI companies, but also making sure that we’re building practices that would be broadly applicable once we can open up that aperture to more models and more model providers, and ultimately, yeah.
Ronan Murphy
I don’t, I don’t know how well I trust the surveys about AI adoption at a personal level, at a business level, because number one, I’m, I don’t spend time reading the questions, so the question could be, have you ever used AI, yeah, put something in Chat GPT, or whatever Claude wants, therefore AI user that doesn’t really follow through, but for serious adoption, for and it’s going to start corporate, and let’s say it’s going to start at business level, that’s that’s what you’d expect, and it can start at a government level, as you’ve outlined, Zach, so maybe putting you a little bit on the spot, but given that you’ve been out a few years already, and the AI office is only really getting up and running in Brussels. What advice would you impart? What maybe do you think you wish you’d done a little earlier or done a little differently to encourage the adoption that speed of adoption within Europe itself?
Zach Whitman
Oh, I mean, I think the conversations that we’ve already been saying, about like getting your data in place, making sure that everything is as flexible as possible, not jumping to solutions, and being really skeptical of the potential ROI, where, where you could potentially see maybe alternatives or like cultural growth before you go in and fully invest, you know, the things that we’ve been wanting to do, and this, this, this exists across the kind of four efforts we’ve been running in parallel. The first one being these, these, these procurement deals, where we tried to consolidate the T’s and C’s to simplify it, to minimize the amount of work required for the agencies, as well as the companies in making these procurements, it takes a lot of time to run procurements and to negotiate one off. It’s much better if you can consolidate those fees and sees where you have data protections in place, and we’re currently on the process right now trying to establish some semblance across the federal complex, and that has been hugely beneficial, because it just means less work for everybody. Secondarily, trying to expedite our security posture, so we have a 20x FedRAMP program, which is meant to allow for faster and more, and frankly, more safer approaches to securing or providing security posture and security controls to these new or nascent, you know, at least in government technologies, and a lot of the companies will come, a lot of tech companies will come to the government and be like, it’s just too onerous to get through the process, there’s too much involved, I can’t, you know, freeze my code base over here just for you to look at it. So, working on a consolidated security platform that can move at this at the pace that the technology is moving was a critical step for us, and I wish we had started that earlier as well. In terms of providing a general platform for adoption, it’s all about the telemetry. If you, if you can maintain data controls where you can see the transactions occurring within your enterprise, that becomes a holy grail data set for your, your operations, instead of offloading all that telemetry to a third party, being able to in-house that, so you can understand all the different network effects that are occurring with the different integrations with AI, that has proved to be…
Ronan Murphy
Could you give us an example of that? So, what you…
Zach Whitman
Yeah, so like, if we establish this general point where we can integrate with third party, so like you have a data interface, like Databricks, for example, it can plug into our API, and now all of our LLM work is using this one consolidated resource, of which we can capture the telemetry, or I can plug it into our RPA team, which is working over here with CFO, or I can plug it into our CRM and Salesforce. Having that direct integration now has built this network effect, or a functional way for us to have an x-ray through the behaviors of the whole organization from this single point. You have a nervous system that’s organically growing as more and more direct integrations occur from this singular point, or you can now start to see everything, and then you can do broad analytics and forecasting, which helps your predictability, which is the big fear here. Like, you can see ramp ups, especially in agentic coding, where there’s just too much consumption that you didn’t forecast for, and now you have to scramble to find the money to keep the work going. So this has given us a far better chance to get ahead of those types of spikes, and ultimately it gives us a broader control set on the whole organizational effort. And then lastly, training has been a big part of it. So we built a community of practice where all federal employees and state and locals join, and we have, we do like regular meetings and seminars, and the team has seen a huge growth in the amount of membership for that, just on general interest, and we’re able to bring our corporate partners in to help explain how they use their products and best practices on topical things like building out agents and things like that, having those four pillars of being able to buy it easily, being able to secure it quickly, being able to integrate it into systems really consistently, and then also being able to train your workforce all at the same time has been really beneficial for us, and I wish we had started earlier.
Ronan Murphy
Well, I mean, you’re busy, it sounds like you’re busy, Glenn. I’ve heard from you before about the challenge in getting the talent who can actually build what what is needed, particularly? I think we were talking about it was in the national security arena, but it is a challenge across the board, and this is this is a chance for Europe, it, and I know, Daniela, you, you can speak to this as well, that you, you found the talent you wanted via various means from various parts of the world, in Portugal, and we might talk about that in a minute, but Glenn, well, what’s can you put a number on it? I mean, how many of the people, how many people out there are capable of building these models in before the model gets good enough to build itself again? Where does it stand today? Dozens, hundreds, thousands, tens of thousands, millions?
Glenn Parham
I mean, definitely not millions, you know. I kind of look at it twofold, in terms of the ability to actually train these like frontier models that are coming out from all of our competitors, and as well as Meta. I mean, it is a quite, from my perspective, a quite small universe. It’s requires some of the brightest minds in the world, my colleague here, who is from Microsoft, and so I think there definitely is a war for talent, and like, where these people are going to concert their efforts from a national security perspective, super important. We want to make sure we are retaining them, that they’re incentivized to stay here to build, that’s like really, really critical. But on the flip side of that, when it comes to AI talent, I think of it like the application layer and the integration side, that’s kind of the work that I did in the government and the Department of Defense, and there’s also a lack of talent there, right? It’s always been hard to get tech talent into the government, we’ve seen over the last year, there’s been a lot of great efforts across the US government to do exactly that, Meta’s, Meta’s has been a sponsor in various initiatives there, which has been incredible, but you really need AI talent that is going to be able to, that’s these people are able to understand the capability of these models, but also like domain specific applications, right, very niche things within, you know the military or where, wherever else, and the US government or other governments as well, they’re able to understand like how to actually integrate these models that makes sense, and and I think that is a rare subset of people, I think we’re definitely making progress on that front, but we can definitely do more, but it’s been pretty like it’s been incredible to see over the past year just how both the demand and supply of that talent has like increased. So I think we’re on a good path.
Ronan Murphy
Well, that’s reassuring, also reassuring that there are other jobs that might be available we can all hope to get in the future, but Daniela, you’ve the experience in Lisbon. There are some specific functional reasons that you can get the talent, because they’re not all Portuguese.
Daniela Braga
No, no, exactly. So, a couple of things. First, Lisbon and Portugal allows has been building a decade of policies to enable qualified talent to come into Lisbon in a very fast visa process, and very exciting tax breaks. It’s 20% income tax breaks, which nobody gets, even here.
Ronan Murphy
Nice, nice.
Daniela Braga
Not to mention that the cost of living is definitely much lower, and the connectivity, both, I mean, both the 5g and fiber, which has been one of the first, even before, before I have fiber in my homes in Bellevue, Washington, Portugal, already had it all over, that’s so, that’s an interesting thing to note now, and also the air flights, the airline situation right now, I mean, we have a ton of direct flights to the US and to North America daily connecting to Lisbon, so we were able to attract faster to Lisbon. When we are hiring people worldwide, it is easier to get them in Lisbon than in the United States.
Ronan Murphy
Well.
Daniela Braga
Of course, for the process of visas and all of that, right? So this has been why, still today, I have probably 70%, 65% of my workforce there, that so that explains that, but the talent part I have to say that AI has been always a multidisciplinary area.
Ronan Murphy
Okay.
Daniela Braga
It’s not just AI. I mean, AI is also a recent field. It’s impossible to have yet the throughput from universities worldwide to get people in AI. You’re bringing together for 20 years people from all over, from all sorts of disciplines and subject matter experts, including myself. My background originally is linguistics, so explains why, and I did a path from linguistics to computer science, and but so it is a multidisciplinary field, more so today, where you see you need lawyers, you need, I mean, our team is, we have a legal department, of course, because we are all about the ethical and legal data sourcing.
Ronan Murphy
Got to hear your all about legal, that’s good.
Daniela Braga
But we have a whole network of subject people, doctors, chemists physicist all stem fields international politics all 500 languages people speaking people speaking 500 languages is not on payroll everyone necessarily it’s a lot of our freelancing group through our platform, our expert connecting platform called Neevo, but you need a multitude of people, and unfortunately, less and less engineers now. Thanks to this rise of AI, I mean, we’re able now to deploy, we have a team of 20 AI engineers just running workflow projects. Workflow is the data collection pieces that we do. We have a marketplace off the shelf offering, and then we have the data collection part of data that we, I mean, the question I always ask my organization, because our clients ask the same questions. Is how can I double revenue with the same head count? This is what AI is doing exactly. And we’re now having, we have an engineering group of 20 AI engineers just launching work. So this is the beauty of it.
Ronan Murphy
Yeah, and I think what we just heard from Daniela from Glenn, it highlights that the race is not run here. I mean, I know there’s there are some gaps, and there’s certainly no European giants, handful of European giants, some of them were in with the Commission President, Ursula von der Leyen in the last few days, talking about how Europe can compete, how Europe can become more competitive internally, and the simplification process has started. What is the near-term future hold? Because there are there are challenges around some of the legislation, the other rules, in terms of we hear it from the firms, we hear it from from some user groups, to you know that are possible barriers to adoption? What’s what’s coming down the pipe, simplification wise? Is it a lot to be determined, I’m sure.
Ruth Bajada
Look, I wanted to say a few words also on…
Ronan Murphy
Go ahead.
Ruth Bajada
…on the workforce, and maybe related to this is that Europe today has become also a place where a lot of people who want to come, and we’ve also seen some people that have also opted to leave the United States and come to Europe. Think one of our member states that is highly benefiting from that is the Netherlands, but we have Estonia in the room. I think the Baltic states today in Europe are very much at the forefront of, again, when it comes to countries like Finland, yeah, you know, you see, and all the other areas, whether it’s biotech, you know, you see…
Ronan Murphy
Malta attracts talent, too.
Ruth Bajada
Yeah, it’s that, I mean, the numbers there will not shift the European numbers because of the size of the country, but, but still, we’re also proud, proud of that, so I think there, you know, we see quite big shifts. I would say that, you know, from from what everybody has this idea of, you know, the European economy is not doing well. I would say, you know, we have some areas where we can really be proud of. That said, I think you also, you know, we’re all worried, we all worry also, we’ve mentioned earlier, China. We’ve all mentioned we are all worried, and all governments in Europe are worried about how you protect your systems. We are, you know, we have a war on the continent. Incidentally, also working very closely with Ukraine is also, you know, pushing also our learning capabilities, and also in tech and tech and defense, but again, when at every single moment, I don’t, I forgot the actual numbers, but the number of attacks, cyber attacks, and hybrid attacks that are taking place on the European continent today, are you know, are frightening, and, and that’s, you know, I think that’s also the next, the next step, you know, whilst you have all, you know, whilst you try and grow, and whilst you try to get companies to scale up, how do you, on the other side, and what you know, what’s role for the European Union is to ensure that you’re protecting against cyber attacks and having, and having all, all that side, plus also making sure that your system is built with trusted partners, and again, there, and when we look at the AI stack and upcoming conversations on that, you know, in Europe, we’re also looking at how to work with the United States to have a safe and Western stack that we can, that we can go with, so you know, so I think there’s plenty to catch up on, but also plenty where we need to work together, because otherwise others are not going to wait, yeah, they will be offering their stack, and they will be offering their stack to countries where we will need to work with for business, whether it’s Africa, whether it’s the Indo-Pacific. So we have no option but to try and make sure that we offer a stack that others are ready to buy.
Ronan Murphy
Thank you very much, Ruth. Few minutes left, gonna open up for questions, and if anyone has any, they’d like to put any of our panel, and we, you just mentioned there, Ruth, the cyber attacks. CEPA published a report recently on shadow war being being run and instigated by Russia, particularly against Europe, and a lot of it is happening in the digital sphere, and it’s something we’ll be talking about. We’ve got a panel again here tomorrow where we’ll be talking about adoption of defense tech and how you can build a future, a future force, and how that makes sense. And Ukraine is definitely a huge part of that conversation, which, although not an EU member yet, is the biggest country in Europe, often overlooked, it is European, so it has a lot to offer in the future. Anyone, just before I step out, yeah, please just state your name and your affiliation and speak up.
Attendee 1
You mentioned one of the things, the four things you’re doing, one of them is training. I’d like to know, and maybe know a little bit more about it. One of the key things, how we’re doing it? It takes a lot of time, I think it’s a change of culture. How has it been?
Zach Whitman
It’s hard, and it’s shifty. What we’ve found is that it’s the there is no single answer, no silver bullet. We have been working on the multifaceted approach, where not only are we doing this kind of community of practice across all the federal employee base, as well as state and local, but also within each agency, working on a number of different steps. There are education pieces on governance, how the thing needs to be approached. If you want to have a use case, you have to go through a series of steps. How to use the thing directly, so for like a chatbot that was like our initial issue that we had was people wanted to use it, they had their own version of it, and how do we bring it in, and how do we have them use it safely? What are the data controls on those things? Are a big question that we’re seeing. So we’re seeing a variety of, like, how do I use this thing, how do I use this thing within my work context, what are the new things that I don’t understand, or maybe are new to me, the ways in which the landscape is shifting from a lot of our employees are inundated with, like, MCP or agent or skills, and they’re always feeling like they’re they’re on a treadmill that keeps speeding up, and there’s a big fear there. And so overcoming a culture of fear was a big problem. And I’d say one thing I would say is having a broad application of the tooling across the entire workforce was a key win for us, instead of selecting like a few pockets where adoption could take place. The literature also supports this, where if you give everyone access to the same tooling, there is a much like a higher likelihood that organic learning will occur between desks. You can lean over to your friend about how did you do this, though they’re more likely to admit that they used AI in the process and not hide behind this or feel like they have an unfair advantage. Those are a lot of the cultural impasses that we are trying to overcome by quickly blanketing the tooling across the entire organization. Then it comes down to keeping up with the Joneses, and as new tools come in, try and incorporate them into a secure system as quickly as possible.
Ronan Murphy
Yeah, human nature still plays a role in adoption, that’s for sure. Oh, yeah. Michael.
Attendee 2
Michael Nelson at the Carnegie Endowment for International Peace. I’ve been doing digital policy for almost 40 years, but I started life as a geophysicist, so I’m really into data. I’m very glad that you spend a lot of time talking about data. Normally, it’s AI means hardware and chips, software, and we have to worry about the people. Data should be getting at least 25-30% of the attention, so here’s the question for any of you who want to take it on. What could governments do to make government-generated data more available and also more reliable? Here, the politics are terrible. First off, politicians can’t see data; they can go and touch a supercomputer, but data is the crown jewel. We must keep our data. The idea of sharing it with the world, that doesn’t make sense to them. So, how do we get more data out of governments? How do we share it?
Ronan Murphy
Who wants to go first? Yeah, go quick.
Glenn Parham
Yeah, very quickly. In my time at the Pentagon, that was one of the actual things that we worked like a lot on. It actually, from what we saw, came down to acquisition reform, which is maybe the most boring thing ever, but ensuring that across the board you have baked into your contracts like a mandate for the different vendors that are playing in whatever space to open up an API, right, and require that that API can connect to whatever other external systems was we found to be the best path forward to actually like making data more available and not as siloed, so.
Daniela Braga
I have to say that I don’t have an answer. I was in the task force for AI in the previous administration for President Biden, where a group of 12 trying to make the data more available for R&D in the United States, starting with the agencies, the federal agencies, which not have a throve, a trove of them of data, and there are there are some websites there that are make it available, but there is not, but nobody wants to give it away, so it’s and and so just that and and then we even this thought about getting an access system where we could, we could basically qualify access to certain people, not others, which, for obviously internal American institutions, and maybe some private companies, and every, we had all it all, but the reality is just setting up that the logging and how many who can access, keeping that up to date, the cyber, the cyber component and risk. I guess we didn’t get anywhere with with that part of the. The situation, I have to say that I, but again, on the private side sector, I mean, obviously we have, we’re doing that in a closed source world, of course, in a smaller scale than all the data in the government, it is not impossible, it’s a federated marketplace, pretty much our partners have revenue shares, and of course, there’s a monetization component, which is not what probably the government will do, but it is not the model we have, the model, just we just would have to get more sources and the government to actually play along.
Ronan Murphy
We’re out of time but we can keep going, give it a few seconds.
Zach Whitman
I used to do the CDO at the Census Bureau, so this is like so near and dear to my heart, hydrating the internet with reliable data and making sure that that’s interpretable to these modern interfaces, of which people are not people are decreasingly going to websites to get information, they’re going to go through an intermediary with an LM. We have an obligation to make our data consumable by these systems, so that they can reach the end user. And ultimately, the model providers do need good data, they need new, novel, original data. This ouroboros of consumption of additional AI-generated content is going to poison these models if we’re not careful. So, using good data from public money to create a public good that benefits not just the individual citizenships or citizens, but also the companies, is so critical to not only the health of the nation, but also the economy, and I think we need to do a better job and invest more in it.
Ronan Murphy
Very quickly, very quickly, with no questions, you’re coming in, the time is up here, very quickly, go ahead.
Attendee 3
I am Dana Linnet from the Summit Group DC, very positive, so take that into consideration for this question. In government, there’s a lot of work being done to keep the models from drifting, because they know that that’s especially in defense. My concern is in the private sector, the model drift is extreme, and even Open AI and these other models would say it hallucinates 78% of the time, people take it as gospel. State of Tennessee built AI and denied 98% of the Medicare folks. J & J just did surgery, surgical AI that, like, pierce people’s spinal cords, and they’re being sued a billion dollars. The question is, so across US and Europe, what can we do to mandate like controls and guardrails on these models to prevent drift, because real harm is being done to people on the private sector, and that is a serious concern. Nobody’s talking about it. It’s all back seven, yay, data centers. Nobody’s talking about the vast harm that can and is being perpetrated, as we are having all this great experience.
Ronan Murphy
Well, there is the EU AI Act route, that is guard rails are very much part of that, but I know if anyone wants to take it.
Attendee 3
But who approves those? Where is the sense of the middle?
Ronan Murphy
I don’t know if anyone wants to take on what’s a massive challenge.
Zach Whitman
I can say that, like, at least in the in the governmental context, because we have executive orders that mandate the transparent and equitable use of AI that we have, the evaluation, the continual evaluation, and the investment to do those evaluations and develop guardrails in the private sector, and that’s the problem. So, the best we can offer right now is that we’re collaborating with NIST, so the center for AI safety center, NIST to publish as much as we can, and to encourage the private sector to do that, but I think fundamentally this is going to come back to legislation. They need to legislate this.
Ronan Murphy
Thank you, Zach. And thank you very much, Dana, for the question. I know we have a couple more questions, you might put them to our panelists as they’re coming off, because I don’t want to keep them up here, and we’ve hit the hit the time, if that’s okay. And thank you very much to our panel, Glenn, Daniela, Zach, and Ruth, let’s give them a round of applause.
