Dela
Hi all, we’re back with the podcast. This is the perfect time for Paul Triolo to join us as he gives us a preview of the upcoming WAIC and walks us through some of the top-of-mind questions we have about China’s AI space right now. I do want to apologize for a bit of the echoing in the background; lesson learned to always use headphones going forward.
In this episode, I speak with Paul Triolo, partner at DGA-Albright Stonebridge Group, about how to think clearly about China’s semiconductor ecosystem, Huawei’s role in the domestic AI stack, and whether U.S. export controls are actually working as intended.
We start with the latest debate around restricting foreign access to advanced Chinese AI models, before moving into the semiconductor stack itself: SMIC’s role, capacity bottlenecks, domestic GPU startups, hyperscaler chip efforts, Huawei’s vertical integration, and why software ecosystems like CUDA, CANN, and MindSpore matter just as much as hardware.
Paul argues that the usual framing- whether China can “catch up” to Nvidia or TSMC is simply too narrow. The more important story is that export controls have pushed China toward a broader systems-engineering response across chips, tools, packaging, memory, software, and cloud deployment. We also discuss HBM, rare earths, remote-access loopholes, the logic behind Huawei’s roadmap, and why the collateral effects of controls may be larger than policymakers expected.
We close on the bigger strategic question: whether the U.S. and China are drifting into an AI race dynamic that raises risks for everyone, and why more direct dialogue — not just more restrictions — may matter most from here. [Paul co-authored a piece here discussing how to navigate the complexities of the U.S.-China AI safety dialog] This is an extremely insight-dense episode, and I hope you enjoy it as much as I did. Thanks again Paul Triolo.
Btw, coming up next are a few episodes featuring founders and execs from hot-listed AI, autonomous driving, and spatial intelligence companies.
To find the previous episodes of Differentiated Understanding, see here.
Every episode, I bring in a guest with a unique point of view on a critical matter, phenomenon, or business trend—someone who can help us see things differently.
Season two will host a series of guests from analysts, VC investors, builders, researchers, founders, and product managers. For more information on the podcast series, see here.
Chapters
00:00 Beijing, Chinese AI models, and why regulators are paying attention04:43 Why AI labs are moving into chip design08:12 SMIC’s role and the fight for domestic chip capacity17:56 Huawei’s capabilities and why it became the center of the conversation26:53 CUDA, CANN, and whether export controls really worked38:24 Can Huawei’s software stack win developer mindshare?51:10 Why China can still progress despite compute constraints57:07 The AI race narrative and why Paul is skeptical of it1:07:46 Why governments still lack the technical capacity to respond1:08:42 Will frontier AI labs eventually be nationalized?1:13:59 The risks of zero-sum U.S.-China AI policy1:20:46 Why more direct U.S.-China dialogue matters
Transcript (AI-generated for reference only)
Grace Shao (00:00)
Hi, Paul. Thank you so much for joining. Really excited to have you on. And I feel like you’re the perfect person for a few of the questions I have prepared in the beginning of the podcast before we get into the actual topic. There are so many things happening, and I can’t keep up with like Twitter these days. So no, so supposedly Reuters reporting saying Beijing is restricting foreign users in accessing Chinese models. Like, what is that all about? What’s your view on that?
Paul Triolo (00:15)
It’s hard to keep up. Yeah, I think in the wake of the fable mythos fiasco, if you will, in the US and the capabilities of these advanced models getting really good and getting into areas like cybersecurity and biosecurity, I think it’s not surprising that the Chinese government is at least considering what to do about open-weight more properly models that are that are that are reaching sort of frontier level capabilities, particularly DeepSeek and Zhipu and then even more recently, you know, Meituan and others. I think my sense is this is just preliminary discussions with the labs about this issue because, you know, even in the US, there’s lots of confusion about what the government’s role should be here in determining when models are released and under what circumstances and how do you measure capabilities. even though the US has been thinking about this for a while, and I’m sure that’s to some degree that’s happening in China, there’s no general agreement on how do you do this. And these companies in China, as you know, are all commercial companies that just like in the US are under a lot of pressure to continue to put out models. and so I think I would not read too much into that. I think that’s it’s clear that the Chinese government is Trying to figure out what to do with about this, but I don’t think they’ve reached any conclusion about which models to control and how to control them. they’re they’re learning from the companies. They’re probably going out and saying, you know, how do companies themselves evaluate these models internally, in terms of capabilities that could be of concern? and what should the Chinese government eventually do? I mean, they’ll they’ll they’ll do something eventually, but I think we’re in the early stages still. as a result of the
Grace Shao (01:57)
Yeah, for sure. I think you know, Zhipu, Minimax, these companies are public listed, like they actually face, you know, just shareholder pressure as well. So it but the one thing, the nuance is Chinese companies usually are a bit more prepared or aware of potential regulatory, I guess, involvement. so yeah, let’s let’s wait and see. Because I read this and I was like, this seems bit counterintuitive, frankly, to the model’s going forward. I think like the deadline was a little out in front here. I think it’s
Paul Triolo (02:49)
Right. I think that the headline was a little out in front here. I think it’s clear that there’s concern as these models become more about what to do about tiered releasing. but this is a more general discussion I think that’s happening. it doesn’t surpr it to people who’ve been following this sector for a long time, the idea that we would be here at this moment, you know, was not surprising. The problem government gov the ability of governments to keep up with the pace of development of the technology is just clearly here, it’s it’s woefully inadequate to the moment because you know within these large AI labs, and I’m now calling DeepSeek and Zhipu, along with Anthropic and OpenAI, you know, the top four global frontier AI labs. you know, th researchers n understand these issues and they’re they’re really concerned about this because P particularly things like recursive self improvement, which is which means models are basically training the not training themselves, but they’re they’re optimizing some of the orchestration or the harnessing that the sort of platforms that these things operate on. You know, that’s been a that’s a growing concern because they’re you know, the models themselves now are able to improve the overall ecosystem without human intervention. Right. and that people miss that, I think, in the in the US with all the fable mythos kerfluffle. The Anthropic released a blog that talked about this, that recursive self-improvement is now sort of part of the landscape. And so that’s also I think part of the concern including in within the Chinese government about okay, you know, Chinese labs are getting pretty good. what should the government how should the government think about
Grace Shao (04:43)
Yeah, definitely. And I think, you know, in China, usually the regulators and the industry actually work pretty close together. so hopefully, you know, people can kind of regulators can keep up, will hopefully catch up on understanding technology a bit faster and better. Okay, so another quick commentary on what is happening in the news. Supposedly DeepSeek and Kai AI are going out and making their own chips. What is happening? What is your high level view on this?
Paul Triolo (05:08)
Everybody’s doing it. Now, you know, in it this is not surprising. US, of course, some of the hyperscalers and more of the hyperscalers like Google and AWS have long d determined that it would be useful have specially designed ASICs, application-specific integrated circuits that are optimized for running certain workloads in their cloud. and you know and optimi na and now optimized for models specif specifically for you know large advanced models for which general purpose GPUs, which is what NVIDIA and AMD produce, may be, you know, may be suboptimal. but this is a complicated issue because to do semiconductor design, you know, this is a whole nother thing than building models. And so for both Zhipu and DeepSeek, you know, this requires building a team of design so some semiconductor design engineers, right? Who now they’re not they’re not a lot of these guys laying around that are not gainfully employed, particularly for designing really sophisticated chips here. So I think it’s not surprising that they want to do this, but I would be again sort of a little bit skeptical that they’re gonna do be able to do this in the you within the next year even. You have to build a team. It’s expensive. use you have to get advanced semiconductor design tools, electronic design automation tools. and then you have to begin figuring out where you’re gonna manufacture these, right? And in China, of course, as we know, because of export controls, these companies are likely gonna have to use SMIC, the domestic foundry. So they’ll be competing with all the other players, all the other GP GPU designers, the general purpose GPU designers like Biren and More Threads, a of these companies that as you know have gone public recently, they’re all vying for this limited capacity at SMIC to manufacture these advanced designs at say let’s say seven nanometers, which is a feature size of these chips. so I think it’s really interesting that they’re gonna do this, but it’s gonna be a challenge on two fronts. One is you know, assembling a team Sustaining that team over time, you know, this is expensive. It’s very expensive. Lenovo tried to do this, for example, at one point. They were considering getting into semiconductor design, but they determined it was it was too expensive and it was going to be a long term drain potentially if you know, depending on the success of these teams. and so, you know, I think if anybody can do it, I mean DeepSeek seems to already have a lot of expertise on hardware and understand the hardware really well. But it’s it but semiconductor design is another whole nother discipline, if you will. And then in China it comes with the added constraint that you have to you’re sort of stuck with SMIC and you know, maybe Huahong down the road will have sub some kind of seven nanometer process. but it’s you know it’s it’s a tricky thing. But again, the trend in the industry is to do this. so in the US you have the open AI, I think just recently Anthropic are both also considering you know designing their own semi
Grace Shao (08:12)
Yeah, they have partnerships as well with other vendors. So help me understand. I think twofolds. One is what is SMIC’s role in China and what is their biggest bottleneck? For them to actually churn out better chips. One thing you said is capacity, other thing you said is access to certain technology, you know, instruments, machinery. The other part of the question, if you can fold into it, is who are the actual players? So, you know, DeepSeek and Zhipu wants to build their own create their own chips, but Baba’s in this, you know, Baidu’s in this. There’s a lot of big tech trying to create their own chips as well. How do we understand all their relationships in the ecosystem?
Paul Triolo (09:06)
Wow, okay, you got a lot a lot in there to so let’s just let’s just look at the demand side, then we can look at the supply side. So the demand side, as I started to allude to there, is pretty heavy, right? So you have Huawei, right? They’re designing the ascends, and those are all at the at least the most advanced node, which is this horrible technical term for the process that is used to manufacture these at SMIC. For example, here. And so Huawei has a lot of demand. And past, Huawei was given most of the capacity at SMIC, just because a year ago or two years ago there weren’t as many other players. Now there are. Now it’s more complicated. But certainly the Chinese government probably also heavily weighed in to have SMIC prioritize Huawei production, both for their smartphone, their Kirin smartphone, and for the Ascend processors for AI. But now in the last year, we have all we have a we have two s two additional sets of companies vying for this capacity at SMIC. One is the G the sort of GPU makers in China, the startups. And this is Biren, Moore Threads, SoftGo, Inflame, Iluvatar. You know, there’s at least there’s a couple more too, but those five are sort of the top companies. And some of those companies were started by engineers from Nvidia and AMD. And so they their designs are really advanced and they are they’re more compatible with the NVIDIA ecosystem, et cetera, et cetera. So they’re all now producing GPUs at SMIC, and there’s allocation issues. I when I was in China recently, I heard that one of those companies had been promised certain number of wafers at SMIC, but then one of the big hyperscalers would come in and offered more money. And so SMIC had said, Okay, well, we’re gonna reduce your allocation, right? So there’s a lot of fighting for that. Th and then the other group that you’ve just mentioned here is are the companies like Alibaba, the hyperscalers, and then now the model developers like DeepSeek and Zhipu that are also doing their own designs. And I think again, as you noted, Baidu and Tencent and Alibaba are more are farther along on this. They have semiconductor design teams which are already producing, in the case of Alibaba, and in the case of Tencent, they have dedicated ASICs. They’ve been doing this for a while, sort of under the radar. Tencent’s a very capable company and has been they really good at this, right? It turns out, but they don’t they don’t they’re very low key on this. and then Baidu of course with Kunlunxin, which is also gonna go public on the Hong Kong market. So the other part of this also is that all these companies of course need capital To function and to do all these designs and to hire these engineers and to and to do all this stuff. So all the in the last year we’ve seen these companies tap into capital markets, particularly, you know, the GPU makers, the new GPU startups have gone public in Hong Kong and in Shanghai. And then you know, we’ve seen DeepSeek, of course, raise seven billion in funding from a variety of sources, and part of that will go to presumably the building up a design team and doing Designing their own chips. and then of course Zhipu has gone went public and its stock is crazy high on the if you look at the valuation on Hong Kong. So it’s a new a new game where Chinese companies are playing this game of d you know, designing their own chips. and then of course they’re all competing for this capacity at SMIC. So now we can turn SMIC, right? So SMIC is this crazy company, right, that for a long time was you know was sort of under radar. but they are under heavy US export controls. So that started in around twenty eighteen, twenty nineteen, when they were they had actually ordered a very advanced lithography machine from ASML, which produces all of the advanced lithography machines, extreme ultraviolet lithography, UV lithography. So they were denied that. They actually ordered it. And then they and then the Dutch government, under pressure from the US government and Wassenaar, which is this interagency or intercountry group, this multilateral group, pulled that license from them. So for the last whatever, six years, SMIC has been using its deep ultraviolet lithography, which is the second best, but pretty good. They’ve been pushing their suite of DUV machines to the utmost limits to try to get to these lower and more advanced nodes. And seven nanometers is sort of the limit, seven and maybe five It’s complicated. Some layers of these semiconductors can be but they don’t all have to be at the most advanced levels. but they’ve been doing something that nobody else in the world has been has done. Now in Taiwan, they did use some for example, that SMIC is using, but they that when they had a access to the more advanced lithography, they went to that because it’s it’s it the throughput is faster. And the yields are better. So SMIC is trying to do something that you know that nobody, no but no other company would have to do because of US export controls. And that means pushing these lithography machines to their limits. they’re having obviously there’s a they’re being successful, but there’s a limit to sort of the yield too that they can do. So for example, the AI semiconductors are much more complicated than for a smartphone handset. So for the smartphone handset, they yield. you know, say ninety over ninety percent. But for the GPUs or the NPUs as actually as Huawei is using, the yields are much lower because these are very complicated and dense chips, right? And so using some of these advanced techniques just it’s just hard to do, right? And it’s and you end up with not as many use useful chips at the end because the yields you know, you’re you can’t do it because you’re you’re you’re sort of pushing the machines to their to their the limit of capabilities. So but this neck but over the next six months it looks like they’re gonna bring online SMIC is gonna bring online more capacity at these advanced nodes, maybe double capacity, because obviously they there’s so much demand for this for this these chips in China that SMIC is responding to this demand and is putting in place more production lines in Shanghai, by the way, at SMIC South to produce to meet the demand for all of these. AI primarily not just AI but mostly AI optimized hardware because of these because of those three batches of companies. You know Huawei is sort of a batch in its in its own right. But then they the GPU makers, the startups and then the ASIC makers. So the challenge then is who decides who gets the capacity, right? and as I said, there I’ve heard a lot of anecdotal you know, chatter in China about that. It’s very complicated. The government obviously weighs in to favor certain companies, but then you know, there’s a tremendous amount of competition. And then finally, the other piece of this not just the logic, if you will, the sort process or die. It’s also memory, right? So for more for the for the AI hardware, high bandwidth memory is a really important part of that because These advanced GPUs are packaged with lots of memory on the on the actual package, in within the actual package, co-packaged, if you will, with the logic. and there, of course, US export controls again have affected CXMT in particular, this case ChangXin Memory, is in Hafei and other places in Beijing. I just saw a big fab in Beijing when was there. and so there but again, it turns out is it to of the export control restrictions on memory is not quite as hard as it is for logic, although it’s not easy. and so CXMT is also ramping up its production of high bandwidth memory, which would then be packaged with those with those dyes. Huawei in particular stockpiled a lot of HBM in twenty four before the US controls were put in place from what Samsung and SK Heino, some of the South Korean producers which are leading In the production of high-bandwidth memory. So anyway so it’s a combination of several bottlenecks that the Chinese domestic semiconductor industry is trying to overcome to meet this demand for these for these AI this AI optimized hardware, whether it’s the Ascend series from Huawei or these other GPUs or these other ASICs that are that are now being designed by the hypo hyperscalers and the model developers like Zhipu and
Grace Shao (17:56)
I appreciate that context. I think for me, like I’ve been reading about SMIC for a long time, but that just really clarifies exactly their role in the ecosystem. And it’s interesting to kind of see how they’re prioritizing certain companies over others. but anyway, I want to double click on Huawei. I think it gets the most heat, you know? It obviously is not And it’s a very interesting company because it’s not just such like it’s not only just like you know pushing out their own models, but they’re a foundry, they’re they’re a chipmaker, they’re a designer, but they’re a bit of everything, they their own hardware. And then they also have obviously then CANN and then MindSpore that trying to compete on the ecosystem side with NVIDIA. So walk us through just Huawei’s capabilities, Huawei’s competitive edge. And then why Huawei was put on the entity list and got so much heat over the last couple of years.
Paul Triolo (18:44)
My God, great question. Great question. You’re really asking good questions here. But you know, there’s some, and I’ve written so much on this. So look, Huawei, if you remember, was primarily a telecommunications equipment company. the in the 2000, 20, that, you know, up until say 2018, 2019. And then Huawei got into the handset business too, right? They built they started building smartphones, they were ramping up. you know, to they were competing and out competing in some sense Samsung and Apple. And then the US put Huawei on the entity list in twenty nineteen, May of twenty nineteen. I still remember where I was when they were when I heard they were put on the entity list. It’s like when I when I remember where I was when shot. and then they were and then they were the even more critical in twenty, they the US added this so called foreign direct product rule, which meant that not manufacture its designs at TSMC basically. And I remember being in at Huawei in 2019 when I toured the their headquarters in Shenzhen and they showed very proudly all of the semiconductor designs they were doing at the most very advanced notes for all of their product lines. But at the time those were you know there was more on telecommunications equipment and also on cloud and on you know servers for clouds, the Kunpeng series of chips, for example. So anyway, so Huawei and the secret there that still important is that Huawei had spent a lot of effort to develop their that design team for those semiconductors, right? So I mentioned earlier how hard it is to do that. Huawei had over the about a ten year period had developed this arm of the company which was designing by the by the twenty time frame, was designing cutting edge chips, you know, on par with like Qualcomm and Nvidia even and some of the other areas. And it was getting better and better before the US basically you know w by putting Huawei on the entity list, HiSilicon was also there. So then High Silicon could not use TSMC. But in the process, Hi Silicon of doing some generations of chips at TSMC, Hi Silicon gained a lot of knowledge about you know both semiconductor design and how to use those complicated tools, those EDA tools, and how to do manufacturing. Because when you work with when you’re doing design and you’re working with a fab like TSMC, you’ll learn a lot about how that happens. So high silicon is sort of the secret weapon and if you will of Huawei. And so after the US controls, the Huawei kept high silicon designing. The designers kept designing Even though they didn’t have a place to manufacture yet, right? And eventually SMIC, working with Huawei figured out how to do all these optimizations of their existing equipment, this DUV equipment, to allow Huawei to manufacture SMIC, even though they weren’t the most advanced process, they were still pretty good, right? And Huawei made all these innovations in terms of overcoming some of the that not being not having access to the latest and greatest tools meant, which was, you know, for things like power consumption and other things. They did they designed around this, right? So that’s the other sort of this is a theme that you’ll that I’m sure you’re familiar with is, you know, the force Chinese companies to do different things and optimize things. This is what happened with DeepSeek, right? And other Chinese companies that don’t have access to all the latest and greatest Nvidia chips. So same thing with Huawei, they figured out how to design around this. Now the other thing they did, of course, which I don’t think mentioned was software, right? Because the US controls, for example, restricted access to Google Mobile Services for their handsets, Huawei invent had to invent Harmony, the HarmonyOS, right? So Huawei had to become a software company, right? Which they didn’t really want to do. Arguably, I talked to the senior Huawei officials and they were like, wow, you know, if we could use Android, why would we go to all the trouble inventing having to invest in and build A whole new operating system for which it doesn’t generate any revenue, right? It’s it’s just it’s sort of a cost center for Huawei. But they had to do it because they were under pressure. And so as part of that process, it’s important to understand that because now we get to the AI stack that you mentioned, the AI era. So Huawei has as a result of US export controls and having to develop Harmony, they have now known sort of the process of how to develop a software ecosystem. and how to get developers to use it, right? Although again with AI it’s it’s harder, right? So you mentioned CANN, the Compute Architecture for Neural Networks, which is Huawei’s equivalent of CUDA, which is the sort of developer and software developer environment that’s critical for NVIDIA. So yes, Huawei is trying to now develop as it did with HarmonyOS. And that was a long process by the way, a long and hard process. It wasn’t easy to do. They had to they and still, right, still China s smartphone companies still use Android, right? They don’t all use Harmony. But Harmony is sort of a cross device thing that you know goes for automobiles. You can use it on your car for the Huawei the Huawei invested EVs. So it’s so it’s a it’s a pretty good system. I’ve seen it when you go with your phone to your car, it’s all seamless, et cetera, et cetera. So anyway, so Huawei knows how to do software development in a complex hardware environment now. And so for AI, which is harder, they’re doing yes, they’re doing can is something that they’re working with. And then also MindSpore is sort of the equivalent of like PyTorch and it’s an it’s a development environment that AI developers use. And so they’ve gone a come a long way on that. In 2020, 2022, 2023, you know, because Chinese AI companies could still use Nvidia, they you know nobody was using Can and Huawei. But now DeepSeek and Zhipu and even Meituan, which looks like they have their million perimeter trillion parameter model on 50,000 ascends. This is something that Meituan has complained. And then now we have Minimax saying they’re going to do a 2.7 trillion parameter model. They’re all working with Huawei to optimize some of the aspects of that software development environment. Now it’s complicated because there’s training and there’s inference. And so there’s different needs for each of those in terms of development. But the again, the US export controls have forced the Chinese model developers to work very closely with Huawei because Huawei is the main alternative, right? And the Chinese government, of course, has been encouraging this. And so, you know, now we’re in the situation where the development environment around the Huawei and the SENS and CAN and MindSpore is better, arguably, than it was even a year ago. When I was at the World AI conference. last year in Shanghai, I talked to a lot of hosting, you know, various capabilities using Huawei Sense and they said that the Huawei system was hard to work with. They would tell me, they wouldn’t tell me this, you know, they I didn’t want to be quoted on this, but they said, you know, they were being told to use Huawei hardware, but it was hard for them to offer the kinds of services they were offering with Huawei hardware the same the same caliber of as with NVIDIA. But now I think that gap is closing. It’s not like Everybody in China’s all the AI developers are rushing to Huawei. but there’s still a complicated mix of both NVIDIA hardware and as we’ll see now, they’ve they’re the Chinese government is allowing I think 10 companies to buy these H200 GPUs, which we should talk about. anyway, so it’s a very complicated and heterogeneous compute environment in China. But one thing we can say with some certainty is the Huawei because of the export controls and because they’re closely with. DeepSeek which is very good at programming the hardware, for example, the that environment is at a stage where probably it wouldn’t have been without the export controls. And so we’re in you know, it’s it’s it’s getting better and better. and because companies are gonna have to use it at some point, they’re sort of d deciding, like DeepSeek is deciding, well, we better put a lot of effort into helping optimize that development.
Grace Shao (26:53)
I think I agree with you and what I’ve been hearing on the ground as well. A lot of the developers saying like if they had a choice, they wouldn’t really leave CUDA just so much better. But if they don’t have a choice, it’s kind of like damn it, I’ll have to just try to learn how to use this. And even if it’s not as good, like we’ll try it’s also like a chicken egg thing, the more developers on it. The better the s the software and the system. But I want to play devil’s advocate here. Like obviously, we all heard the Dario and Jensen like interview. Right. So like we don’t have to get into the details of that. But part of the argument. But part of, you know, what you just talked about was like, you know, high silicon kind of got shafted. They couldn’t get access to certain machinery. You know, obviously, right now we can say that. Objectively, factually, Chinese chips are probably not as good as the leading chips globally. So thus some may argue exp export control worked, right? Like so what’s your view on that?
Paul Triolo (28:02)
Wow. Okay, that’s a that’s a rather large topic. So it sort of depends on what you mean by work. so look the original goal of the export controls as sort of articulated in the you know federal register notice in October twenty two was originally related to s you know to sort of military other sort of nefarious end uses of right? but the real driving force, if you will, was really the this idea slowing Chinese companies’ ability to develop frontier models down so that the US would get to some advanced level of AI first, right? And so if you just look at that and you look at say Zhipu releasing GLM five point two, that’s not as quite as good as fable or mythos, but it’s pretty good, right? And it’s the gap between le the le the leading models from anthropic and openai and the leading models from Zhipu and DeepSeek and other Chinese companies and Alibaba in particular and now you know Meituan and even Xiaomi and Minimax, you know, that gap is still there, but it’s not really is it months, is it a couple of months? So if you’re going to argue that the export controls worked, then you know, is the does a two or three month gap even matter now, right? so that’s that’s one way to look at it, right? Now, if you look at it in terms it make did it reduce the ability of Chinese companies in the semiconductor industry to manufacture advanced GPUs, for example, on par with NVIDIA, of course it worked, right? Nobody would argue. that it did that didn’t happen because if you’re gonna restrict exports of GPUs and you’re gonna exp r you know restrict exports of critical tools that are used to manufacture those GPUs, of course you’re gonna you’re gonna slow them down. But then they then you have to say well how has Chinese how has Chinese industry responded to that, right? And what are and what are the costs of that for US companies, right, for example. And then you have to look at what is the retaliation from China to those export controls. So you have to at least look at it, look at the picture more broadly than just, you know, did the US slow down Chinese model development? Arguably there, the jury’s still out on that, right? Because I would argue that they haven’t the slowdown hasn’t really been that significant. If a company like Zhipu, like who had heard of Zhipu like even a year ago, right? if they can release a model like GLM 5.2, okay, wow, that’s a that’s a frontier model, right? It’s it’s matching fable in some benchmarks. Okay, so it’s clearly somewhere near the frontier. How close we can argue about and a lot of that is complicated, depends on the benchmarks you’re using. But in the in the semiconductor industry, then the you have to look at that in a little more depth. One thing that I’ve written quite a bit on of course it’s forced Chinese toolmakers to work With the with SMIC and Huahung and some of the other manufacturers, CXMT and YMTC. And so the overall level of capability, for example, of Chinese, the Chinese semiconductor industry to do stuff domestically has gone way up. So those toolmakers, for example, NARA and AMEC and Piotech, they’re now competing outside China with US companies in a way that was inconceivable in 2022. And so what happened, of course, is As a result of the controls, the US companies had to pull all their people out of those fabs in China. guess what? US competitors, US company competitors from Japan in particular, and also Chinese domestic companies had got access to that equipment. And they learned things from that equipment that they wouldn’t have learned if the US companies had been in control of that equipment. And so that’s one just one of many examples of sort of the way you have to look at this if you’re gonna say, did they work? Because as a result of the controls, the US now US companies now have competitors globally for the in the tool making sector. So for example, NARA and other companies in China have been qualified for TSMC to provide tools to TSMC and to Intel and Micron, right? and so now US companies face bigger competition. And the ability of China’s semiconductor industry to pr to eventually produce more advanced chips has gone way up, right? So because the that semiconductor part is complicated. You know, the idea that the US, example, could use controls to forever keep Chinese companies from developing n capabilities sort of, you know, it’s unrealistic, right? Because th this is a this is an applied science. And so the ar the US argument was this is a choke point that we can stop China from doing, but no, China’s designing around that because there’s many ways to do things, right? There’s there’s more than one way to do to develop a tool, for example. And The industry has pursued many different paths and over the years, some have more commercially viable. Those have been the ones that dominated, but now China is pursuing other ways to do things. And so you know, that’s that so like hu like Huawei, just a quick example. So Huawei just you probably saw a couple those last month, I think. They came out with this Tao scaling idea. And so is the re reduction of feature size is to increase the speed and the and reduce the power consumption of these chips. And so that’s Moore’s law has held for a long time. But now we’re run, you know, the industry is running up against just the limits of physics in that in that regard. We’re down to you know one nanometer, you know, really small feature sizes. And so Huawei is saying, okay, well there’s maybe another way to do that. We can we can use a sort of three dimensional structure here to also to move the components closer together and to reduce the time, the latency between signals going to those components. And so that’s not new in industry. This approach has been used before or you know, people have been looking at this. But Huawei is now putting a lot of effort into the actual tools and the technologies to actually do that at some kind of scale. Now the jury’s still out on when that will happen. They’re saying by 2030, for example, they’ll have a s a feature size that will be a system that will be of like a 1.5 nanometer system. And again, the other thing to remember here is that it’s not now just about feature sizes, it’s about sort of the entire package of the system, right? It’s about the memory, it’s about the interconnections, the optical interconnections between the GPUs, where Huawei, for example, has a lot of knowledge of optical interconnectivity. And so you can’t just you can no longer just look at the individual sort of feature size die to die and then and then determine that you know China is ahead of the US or US is ahead of China. So it’s a more much more complicated calculus. And I’ve written about this quite a bit. But you know, I wrote the I think two years ago I noted that you know that now we were in a different ballgame. It was really systems engineering at a at a higher level that’s going to determine you know the capabilities. And here again, Huawei has some significant advantages. so anyway, so the long worded answer to whether the export controls worked is well, yes, of course they worked at some degree to stop and slow down Chinese industry. But at the same time, they’ve accelerated key parts of that industry. And then finally, I would argue the rare earth issue, which by the way I live every day because we’re trying to help companies overcome some of the issues around many licensing and other things, you know, that has been a huge thing because that was directly responsible in response Gallium, graphite. And then of course in April last year, the controls on heavy rare earths and magnets, right? And so those are still, you know, with us. Just today, you may have seen and yesterday, and Nikkei had a story about Japan, Japanese companies who which have been cut off from rare earths in Jan starting in January. They’re filing with the Tokyo stock exchanges are indicating they’re because they’re running out of these critical materials. And all of that result of the of the US export control regime and China’s response, which is to put in place this very strict licensing regime around rare earths, the way, are key inputs for the semiconductor industry too, right? So yttrium, for example, is used to line etching chambers. and most of all those machines I mentioned, DUV, UV, they all use lots of rare magnets for various purposes. every ch semiconductor produced in the world, virtually every one, is touched by a plasma, which is this gaseous, you know, material that’s controlled by Chinese rare earth magnets, and the chambers where that plasma is contained are lined with Chinese rare earths materials. So in other words, the export the US ha put in place have resulted in this very serious response China. That we still are in the middle of. We don’t know how it’s going to come out, but it’s already had a huge impact on the entire supply chain for the semiconductor industry. And not just semiconductors, but of course and power tools and any industry that uses these materials. So anyway, so the disruptions that caused by that are huge. And so when you’re looking at the cost, so we’re looking at the costs and benefits. Did the US slow China’s AI development? Yes, degree, but Jury’s still out on how much. And then if you look at all the collateral damage that those controls cost, you know, those are stacking up and there’s no end in sight right now. So that’s but my view is always, you know, you can you can I agree that the controls work to some degree, but then the question is, you know, what was China’s response both from an from an industrial point of view in terms of working around the controls, and then what was the collateral damage created by the controls and that is that is considerable, I can tell you.
Grace Shao (38:24)
Paul, I love interviewing guests like you because I was gonna follow up with like HBM in the whole picture, how that affects it. You already answered. I was gonna ask you about inferencing versus training on Huawei chips. You answered it. I love you give the full picture. but I wanna ask, what is it like you talked about collateral damage and how these like industries kind of came out because of export controls? Now, how do we understand actually potentially CUDA? I sorry, not CUDA can. Taking some market share, I wouldn’t say lead at all, but some market share away from CUDA and potentially courting more developers globally, maybe beyond just China. How does that new ecosystem and operating system meet work? Because I would challenge and say harmony at this point is still nowhere close to being a dominant operating system, right? So despite you making the point that they obviously had to go around it and create harmony and it does exist and suffice for their own hardware ecosystem. It’s not a leader. How do I understand that?
Paul Triolo (39:23)
Yeah, yeah, that’s a great question. That’s a great, great question. So, you know, this is a this is an older question remember, you know, the China didn’t the Chin there was no Chinese operating system, you know, for j like for PCs back in the day, remember? so we had things like Red Hat Linux, you know, or Red Flag Linux, right? Which was the which was a sort of source Chinese version of Linux that was touted as gonna you know, that was gonna be sort of the Chinese version of Windows because of course China has been dependent on Windows for a long time and still is to some degree, right? And so this big the big this issue of sort does how does how does China how does China develop alternatives to existing dominant software ecosystems like Windows or like Android CUDA. You know, is a is a is a really good question. And it’s and it’s sort of it’s a complicated issue because each of those has a different a different dynamic there. And it’s and it turns out to be really hard, right? Because developers and I know this from installing CUDA software environment on my home computer where I have a I run an RTX 4090 NVIDIA GPU, which is export controlled to China, but I wanted Seek. a deep seek model on my home my home system and I had to install all of this development environment which is very complicated which included you know PyTorch and CUDA and all these things, right? And so when you’re a developer and you’ve been working with all of these things for many years, the idea and somebody tells you, you’re gonna have to now switch over to this other system, which you don’t know, and you don’t know the limitations and the and the strengths and the weaknesses of that be like, Like really? Do I have to do that? You’re not gonna want to do that. You’re gonna resist, right? and so this and same with Nope when Chinese companies were using Windows and somebody said, Hey, here’s red flag Linux, which of course wasn’t very good. and you by the way, you can’t run all your Windows applications under Red Flag Linux. so you’re gonna have to run, you know, weird open source versions of all of all your favorite programs. Again, you know, I did that for a while. I actually Linux exclusively for a while, but then I ended up coming back to because you know, there was certain things I couldn’t do. so same thing here and same thing with Android and Harmony. So it’s a it’s a but as I said, when like when Huawei first started on Harmony, you know, they had a hard time convincing developers in China to use Harmony. But now you know I think that process is pretty far along and other I re just recently you know other companies are starting to use Harmony and so Event it depend and again it depends on their business model. If you’re s a Xiaomi and you want to sell handsets outside of China, you’re probably gonna go with Android because you can still use Google Mobile Services, right? I mean it was really a d a devilishly clever thing for the for the administr for the for the Trump administration to control access to Google Mobile Services because that really killed Huawei’s business China. And that was a key source of revenue, by the way. So that was not an accident, right? Like wh at one level it was like why should they do that, right? It’s not military technology. It’s it’s it’s you know, YouTube and Gmail, right? But the reason was they really wanted to kill Huawei’s handset business. And so that’s why they targeted that. But other Chinese companies can still use Google Mobile Services. So Huawei in that case is operating in a in a in an environment where Android is still out there, they haven’t Android is still available in China. So they have a they’re competing against they’re still competing against Android. Now CAN, it’s tricky here because in addition to Can, as I mentioned, those other GPU companies like Biren and others, their develop they have their own development environments. And those development environments are more compatible with And then in addition to that, Huawei is trying to make CAN and the whole environment more compatible. with CUDA. So the idea is that you know the difference between the two. If it was here three years ago, now the difference you know, is less. And so it that willingness of the developers to move to environment easier as you sort of reduce the differences. And so and it’s hard to gauge exactly where that is, right? Because each and each company is different. So DeepSeek, example, I is different in the sense that those guys were programming the hardware directly, right? So if you don’t if you if you are really good and not that many have engineers that can do this, you don’t need CUDA. You can program the hardware directly, right? You CUDA is sort of this intermediate layer that makes it easier. It’s a bunch of libraries and it makes it easier developers to train you know use the training environment. But if you know how to program the hardware directly you don’t need CUDA. So anyway, DeepSeek is sort of unique in that they were really good at the hardware. and so that’s why it’s important that they’re working with Huawei because they understand you know the sort low-level way that these systems all work together so they can help Huawei to improve the ability of the capability of Huawei’s hardware development environment to more to be with CUDA. And so I think we’re in the process of having that happen What’s probably gonna happen in China is gonna there’s gonna be a sort of s system where Huawei will be and CANN will be used more for inference on the inference side to inference and some and CUDA and NVIDIA will still be used to some degree on the training side. Because remember, it’s complicated. Right now, Chinese companies can still, for example, use remote access to services like in Japan and Southeast other places. to train their models. And so they can continue to use the NVIDIA development environment for that, right? And then and then when you get to inference, they can then use Ascend and they can run that on there and they can they can optimize using CAN to run on the on those on the on the for on the inference side. So we’re in this sort of a weird world where you know there’s the developers haven’t all switched over to Huawei and they still don’t really want to. But more and are there’s more effort And ease that transition CANN with CUDA. And so where we exactly we are in that is hard is hard on any given day is hard to say. But clearly, as you noted earlier and I and I tried to stress, the problem is that for the long term, the Chinese government and these companies don’t know what the US policy is here. So we just saw that hundreds, you know, maybe two hundred thousand H200s will probably be approved by government. For companies like ByteDance and Alibaba and Tencent and others to buy, right? Okay, so they buy those. They can use those. Those are really good for inference. They can just, know, they can they have a lot of demand for their for their services. They can they can throw those in and they can be used for inference. They can also be used for training if you know what you’re doing, right? You can tie a lot of those together. but what next, right? So what is the US government’s policy? basically, under the influence of Jensen Huang and agreed to stop. Forcing NVIDIA to downgrade their chips for a set sale to China. And so Trump said, okay, you can we’ll allow them to sell, you know, not the cutting edge, but something a couple generations behind the cutting edge. So hence the H200 class GPUs. But what’s next? So if you’re if you’re a Chinese company, you can’t count. Any other in the world doing AI design can say, okay, I’m gonna, I’m gonna, I’m gonna upgrade my cluster from H200s to Blackwell, and then I’m gonna upgrade to Vera Rubin, which is the next one, and then I’m gonna upgrade to Feynman, right? So there’s a roadmap of updating your hardware cluster. China, you know, what’s what comes after the H200s? So therefore the pressure is to and this is why Huawei eventually issued a roadmap, right? Huawei had never done a roadmap for any of this, but now Huawei, because dynamic, had to come up with a roadmap. And so that’s why they have the Ascend 950 you know, the nine the nine twenty and nine fifty. So now they have a roadmap out to twenty thirty or tw twenty thirty one that’s their roadmap for upgrading their the domestic processors. So if you’re a Chinese company, like ByteDance or like you know Alibaba or Tencent, all the leading players, you have to figure out a very complicated equation which is how do I keep my core developers who are using CUDA happy, using some hardware, either in China and then how do I gradually transition to using domestic hardware for some workloads, right? And again, these companies can they can run different workloads on different systems depending on what the need is. and so they’re and then at the same time, you know, maybe they can get some GPUs from REN or you know Sofco or some of the other smaller players and run those are primarily inference workloads. And so but they can but those are those are really good, you know, those are very, very capable. GPUs. So they can run some stuff on those and experiment with those. And those are gonna be easier because those are more compatible with the Nvidia ecosystem. So anyway, so have a very complicated hardware environment to navigate compared to Western companies. You know, like OpenAI can just keep its clusters depending on you know how many GPUs it can Nvidia and AMD. so it’s it’s it’s a very interesting and heterogeneous situation here. Where it’s different than Harmony because Harmony is, you know, it’s still developing the developers develop apps to run on Harmony, right? And so you have you have that piece. it’s like a it’s there’s inference and there’s training and there’s a lot of different things going on here. there’s runtime stuff that you’re doing, there’s harnesses, which are the ecosystem around these models that make them capable. so AI development environment is much more complicated than har than for a mo just a mobile operating system. Will. But again, the export controls and the uncertainty of policy really they’re there, right? And you know nobody has said that eventually the US will allow black wells to be exported to China, for But we’re still in this weird Chinese companies can and access those restricted semiconductors outside of China. They can they can run training workloads in Japan, right? And so that loophole may be may or may not be closed over the next year or so. but in the meantime, you know, Chinese companies have options and each company’s different, right? Because DeepSeek, for example, wants to have its hands on the hardware. So they don’t they’re they’re probably not gonna use anything overseas. They wanna have the actual hardware because that’s what that’s what they do. But Alibaba and ByteDance and companies, the hyperscalers that have data centers China. You know, they’re probably gonna they’re they have more options, right? And some of those H200s I think will probably go into could go into data centers outside China too. and then NVIDIA is selling the CPUs now are also really important for some of this. And so there’s no controls. It’s a weird loophole, but the Vera CPU, which is used with the Vera Rubin GPU architecture, can now be sold to China NVIDIA just this in the last couple of weeks is marketing that to China. That’s a very capable CPU, which could paired with other accelerators and used for AI training and other things, right? So that so the compute environment in China is really complicated by the because the are there, but they don’t cover everything. They don’t cover remote access. They don’t cover CPUs. and so Chinese companies now have some options here, but they’re, you know, but again, it’s like what is Here, right? It’s much more complicated for a Chinese AI developer than it is for OpenAI or Anthropic, and that’s that’s sort of the
Grace Shao (51:10)
Absolutely right. I’m really glad you brought up the H200s because I was gonna ask you about that. And it’s very interesting to learn that the CANN like system is trying to become more like CUDA and it makes sense if you’re trying to entice people to move over. Okay, I have a question, I don’t know how to ask it because I’ve heard it asked in both ways. Some people are saying, therefore, why is China still lagging behind if they’re capable of still getting access to certain ships and they’re so talented, right? Or the question could be asked in different kind of framing, which is why like wait, I just asked why are they so behind, right? Others are saying, why are they be able to catch up, play catch up if they’re so limited to, you know, generations ago. So, like, you know, the same question is basically being asked with different framing. At a high level, how do you view this right now? Because frankly, going back to your commentary even on open AI being able to just keep spending and keep purchasing. The most frontier GPUs, then the question is, is that price even justified if you can get almost frontier near frontier with, you know, four generations ago GPUs, then why do you need to spend so much on the latest, right? Like there’s a lot of discussion around that. Is the CapEx kind of justified? I guess this is a big question. See how you want to answer it. It’s complicated. Yeah no
Paul Triolo (52:25)
We should probably do a whole show just on that, because it’s complicated. Yeah, no, that’s a great question. And you’re you’re as you’re you’re really good at asking, you know, the really tough questions here. So I mean the and I think the d the difference then is that in the US, this idea of scaling, right? The scaling laws still hold. So the more GPUs you throw at training, the better the models will be. You know, that’s still sort of the view in the US. And it turns of that may be true to some degree, but there’s a lot of factors, for example, besides scaling that make models capable. There are these, there’s the harness thing, right? Which is the which is the ecosystem, the orchestration around the model. That’s really important in terms of the performance of the model. What tools can the model call, right? There’s a whole huge effort, you know, to standardize the calling of MCP protocol. which is used to connect the model to other applications. I just hooked up, for Claude. I gave Claude access to one of my brokerage accounts. And it can go in there and pull all the data on all my investments and analyze it, right? and so it does and it’s really good. And it learns, you know, more about you know certain other topics. A little bit well I have I had to sign away I had to tell the brokerage made me like you know sign away all the rights to any you know that happened.
Grace Shao (53:38)
That sounds so risky, Paul, and you’re so brave.
Paul Triolo (53:50)
So anyway, but the point is that the raw model and the scaling and the GPUs, it turns out that there’s more the scaling does still hold, right? I mean, Dario from MADA, where of course the CEO of Enthropic, you know, he was like discoverer of the scaling laws. And so you the major US labs like OpenAI and Enthropic and Google, and you know, there’s still the sense that the that the more GPUs throw at it and the more training you’re doing, you’re gonna get better models. But it turns out that like DeepSeek and others, there’s you know, through optimizations, because they don’t have access to unlimited compute, they figured out ways to optimize the and to enable them to run more cheaply. and that and that’s that’s affected the diffusion of the models. So if you’re when you talk about you know who’s ahead, there’s sort of raw model capability is one thing. And then there’s like who’s using the models, and everybody, it turns out that everybody doesn’t need the most advanced models. To run to run really useful applications, right? And so that’s where the Mabel fable and mythos thing, it turns out that you know people are now worried that the US government will, for example, cut off access to these models. And so why wouldn’t you use an open s a really capable open source model from China like GLM or Moonshot? Kimmy is very popular in the US. And you see in these recent reports that you know Coinbase and Microsoft and all these US companies are considering or using Chinese open source models in production, right? And so the question there is, you know, those models are exactly as good as the US models, but they’re pretty good, right? They’re good enough. And so it’s is the question of who’s winning the sort race, you know, is sort of less material in some sense because of these other factors, right? And so it turns out that the that both the model capability, once it’s near frontier, it’s good enough to run most things, right, that need. Some companies will still want to have the most cutting edge model. And also they’ll wanna have the issue of like their where their data is, right? They’ll wanna trust the company that’s that’s running their data whether it’s through an API. they’ll wanna trust that company their data. and maybe they don’t wanna they don’t wanna do that you Chinese model cloud, but they might be willing to run it on premises, a Chinese open source model and build on top of that. And that’s that’s that’s also what Fable and is sort of forced issue. Now companies are thinking, why do I want to give all my data to Anthropic or OpenAI when I can run a very good Chinese open source model on my own infrastructure and I can control the data and the and the security of that of that of my you know my business model. you may have seen Alex Carp’s sort of rant, people some people called it a ramp a couple days ago where he was talking about that issue where he was basically saying that, you know, don’t want to give all their data over to these model developers because then those model developers will compete with them for certain things. Which what’s happened like what
Grace Shao (56:41)
Yeah, they will eat their lunch instead. And people also have a misconception that like when you use a Chinese model, it’s not like you’re giving the data to an open source Chinese model because you actually can self host that model I in your home countries. Anyway, I’m gonna start wrapping the conversation. I want to go big picture. Last okay. Last question on this. The dominant narrative, DC, is that, you know, AI policy circles, you know, circles often talk about, you know, whoever achieves AGI first, whatever that means these days. Will gain a decisive strategic advantage and ability to reshape global power. So it’s very, very scary. You know, how do you view this? Because through our conversation, what I’m hearing is that both sides are, you know, cautious, both sides are healthy and skeptical, but both are putting regulatory pressure, whether domestically on the companies or, you know, on protecting them from, I guess, foreign actors. Is this actually conducive for the future? Like how should we kind of view this? Because It also feels like from our conversation, a lot of these export controls, protective measures are not actually working or actually good for the industries domestically. So just a high level, like how do we understand this? Yeah.
Paul Triolo (57:47)
Yeah, great question. I think level, my concern, and I’ve written quite a about that, you know, if we race argument that the US is indeed to prolong the gap between US models and Chinese models so that when we reach something and I think AGI, I think we’re already at We’re already the models already passed the Turing test, right? But we’re talking about like artificial superintelligence where models are you know self they’re self-improving and they’re they’re coming up with really amazing new designs or weapons designs. If you look at the AI 2027 scenario, that’s sort of that’s sort of how people are thinking. Now I’m skeptical of that scenario because I think you know we’re still away from these models being to do you know the design super weapons and take over everything and so that one side who gets there first wins And then can kind of lord it over the other side. You know, this is essentially what Dario saying in his essays, like the machines of grace, and the adolescence of technology. He has said basically like AI, d democratic AI needs to win so that then that can be used to sort of force regime change in China, right? Or force authoritarians to sort of, you know, kawtow, if you will, to Western the Western governments. But I think that’s a That’s a I don’t like that framing because I think you know we’re not gonna wake up one morning and have that capability. It’s gonna be a gradual thing. and then the real issue is how governments deal with this? Like this whole issue of fable and mythos has forced the issue to the fore of how do governments deal with even just capabilities? This isn’t super intelligence, but this is like really good capability to detect vulnerabilities and software that exploited. And we don’t even have a framework for that, let alone for something more advanced intelligence. How would the government and industry work together on that, right? So there’s a couple things. One is, and I’ve written I just wrote a piece in Cairo Review about when does the government think about nationalizing the AI labs, right? Because people are using these analogies like these are nuclear weapons, right? Even though AI is not nuclear weapons. And I think it’s a very dangerous analogy. But people are saying, you know, this is a technology developed in the private sector, previously, weapon systems that were very capable were developed by government. And here we have a private sector ca capacity that’s that’s starting to edge towards weapons systems with cyber capabilities, for example. What is the gu how does the government do fit think about that? And the mythos thing, frankly, showed how unprepared the US government was for this, right? If you’re in the industry, you know that you knew that this was coming. I we talked about this last year at the Paris AI conference, right? You knew that this capability was coming, but nobody in the US government, the Trump administration was just saying innovation, innovation. China’s in the same way, right? How do you balance regulation and innovation? They want the companies to compete. So there’s no regulation. now Mythos and Fable have forced the issue of like, my God, well now we need to have some government role in determining, you know, how to test the models for certain capabilities and how to determine what’s a covered model and what should the conditions be around w how that model is released. And at least with Fable, we saw company, in this case Anthropic, have to, you know, put guardrails around the cyber capabilities of that model. And now they’ve finally the government has allowed them to release Fable. But that’s not there’s still a lot of questions around that, right? So the problem is if we’re in this, if we accept this race idea, then we’re never going to get collaboration between the US and China here, which I think is really dangerous because then, you know, malicious non state actors are gonna get access to this capability. and then, you know, the implications of that are really, really serious. And so the problem with the race idea is that it forces everything is it that then becomes distrust. The US distrusts China, China distrusts the US, you know, the US is gonna ban open could ban open source Chinese models. China could you know r restrict the release of open source models because they don’t want to contribute to The US developing capabilities. So we’re gonna get if we get into this race, then it’s a bad thing for everybody, I think. So we have the US China AI dialogue, which is coming up hopefully after the World AI conference in Shanghai, which I’ll be attending next week. and you know that’s gonna I think that’s the last chance. It’s the last chance for the US and Chinese governments to say, okay, we understand we don’t trust each other, but this is a threat, the threat of you know uncontrolled access to these models. It’s a exactly. It’s so it’s the last chance for the for governance to recognize that. And I think we I think that’s the mythos fable thing. The good news is that really drove I think this agreement in Beijing and Mart and May to between the t the two presidents to start talking about this. But it’s a complicated issue because ha you know, you got what are what is the goal here? How are you gonna agree on both sides to some limitations on this, right? And then how do you how does this translate into eventually a global agreement on putting guardrails around frontier AI models. It’s it’s it’s the problem is the technology is developing so fast that the ability of governments to keep up with this and come up with, you know, credible and viable structures to put some controls around this, you know, it’s really tough because there’s just not enough expertise in government. It’s going to probably have to be an independent, private sector led effort to do this. And this is This these are the kind things that are going to be discussed in week. I’m on a couple of panels, including some closed-door panels, that where these issues will be discussed. Now nothing’s gonna be decided next week in Shanghai, but I think the level of the discussion will be much higher because of fable and mythos and because the Chinese government is kind of freaked out about this. and you know, and the good news is that at least at some point the US and China will eventually sit down and try to begin this. Scott Bessent is gonna head up the US side and Vice Premier He I just did a piece with Alvin Graylin that you’ve probably seen in on the ASPI website, which tries to look at who’s gonna participate in this from both sides because there’s lots of lots of equities and we saw the that whole issue become complicated just in terms of deciding what to do about Fable. it was good in the sense that the governments to have a serious discussion of what are we going to do about this, right? So that’s the good news. But when I was at the way the finally, when I was at the World AI conference last year, I think it was Stuart Russell, my good friend Stuart Russell, who said, you know, he had talked leading CEO of a of a US lab, and he had said the best thing we can hope for in the next two years is a Chernobyl style event, right? Now think of what that means, right? This is the head of a lab. admitting that you know AI could lead to a very bad outcome here, right? and so this is where we are here in the summer of twenty six, where US and China both have leading labs. Governments don’t seem to know how to get a handle on what to do about that. But the US and China have to talk about it. Because if we get into this race, if we if we if we if we basically give up sort of say, okay, it’s gonna be a race, you know, and then th what will happen is something bad will happen and then and then people will say, my God, now we this. and you know, some people think that Fable Mythos thing is good because there were there was no bad event, you know, no loss of life or no nothing. But it did sort of force people to realize, okay, now we need to do something, you know, that’s a good thing. But still the political pressures and all these things we’ve been talking about, the export controls and everything, you know, there’s no there’s just no trust on either side. we’ve dug a deep hole in terms of and so Digging out of that, as you say, you know, to for the benefit of humanity is gonna be r a real challenge now. but you know, hopefully, as I say, in the next couple months, we’ll know better where that dialogue is gonna go and where China’s gonna be, for example, coming out of the World AI conference. They might announce the World AI Cooperation Organization, for example. I suspect they will announce that. Xi Jinping is coming, just to show you how important this issue is. Xi Jinping is coming to Shanghai. So the security arrangements around this conference are nuts. I’ve just been trying to figure out where I’m gonna be on different days. so that shows you how important it is. If Xi Jinping is coming, it’s important. and if Xi Jinping has agreed with President Trump to discuss this at some level, that’s that’s good. That’s good news. But as I say, I think this is like humanity’s last chance to get a handle on this because you know it’s the US and China have to agree. Everybody else matters, you know, there’s a big safety community, there’s other capable model developers in other places, but really the US and China are where ninety percent of the action is. and so if there is no agreement between the US and China, beginnings of an agreement around this, then you know, then all bets are off. And so I think this is a really the next couple months are really critical in this in this arena. And you know, the and that’s the technology continues advance and recursive self-improvement kick in. And as, you know, these things, you know, that’s not gonna stop. There have been all these efforts to say, hey, let’s let’s stop until we figure out what to do, right? Let’s pause, right? Who’s gonna pause at this point, right? I mean, those a year a year and a half ago that all these scientists, including it from China, signed on, like, we gotta pause, six month pause. The hard part is if you pause, you know, how do you decide when to start up again? Right? it’s it would be you know impossible. So nobody’s gonna agree to a pause. So therefore we need to agree on A minimally viable framework around governing these advanced models. And that’s what the goal is going to be in the next couple months. But you know, it’s it’s it’s really going to be hard because of the government the lag in capability in government you know so the mythos thing highlighted both need to do something but also wow like it’s the people who really understand these issues are still limited in number
Grace Shao (1:07:46)
Yeah, we need more technical people in the government. Like actually every government. That’s the thing. Because this technology is not for the laymen to understand, frankly. Like you need someone who’s technical to understand it. But I think, okay, on that, like I’m feeling serious FOMO. I was planning on not going, but maybe I’ll go up. It seems like everyone is going. I was speaking to Alvin Graylin this morning actually. We’re working on a piece together. So it’s very interesting. I’m glad he’s he’s potentially going, you know, Ray Ma’s going, a bunch of people in the circles going. So You know what, like I think you’re right. Like I really do hope that something positive comes out of this. It’s just seems like it’s really hard to regulate something when regulation takes so much time. There’s so much bureaucracy that comes with it. And then on the other hand, like exactly to your point, AI doesn’t sleep. I was joking with my husband, I was like, I just want to summer. And he’s like, AI doesn’t summer, you can’t summer. And then he like being a tiger husband there. But you know, the reality is no one’s gonna stop right now, right? And like
Paul Triolo (1:08:28)
Right, right. I want three months off from all this, but catch up.
Grace Shao (1:08:42)
There’s a commercial interest and there’s also the com competition competitor like I guess even spirit in these researchers at this point. So it is gonna be incredibly hard. Yeah. So are these things gonna be nationalized? Do you think these like I mean, the irony and all this is like a look at my Cairo Review article? Yeah came out last month when I tried to lay
Paul Triolo (1:08:50)
Right, and we also have massive IPOs coming up, right? We have anthropic that’s the other complicated Well, take a look at my Cairo Review article that just came out last month, and I tried to lay out how both the US and China view this. And I think y arguably already, you know, there is some there isn’t national you know, is gonna happen in different ways, but some level of nationalization is gonna have to happen, right? We’re already talking about open AI g you know, pr that the government taking a taking a investment or taking a share. OpenAI. So that’s kind of a there, right? so I you know it’ll be it’ll be different than nationalizing other industries, right? But yes, I think at some point it’s it’s hard to see the government leaving this capability in the hands of the private sector fully, right? because a important capability. And as we get closer and closer to more advanced you know AI and The idea of like loss of control, what happens, what happens if we lose control of the AI? all these things are out there. And so I think, but again, we can’t even figure out basic government role in, you know, how do we how do we determine what is a covered model and who is who is equipped to test that model, right? there are very there’s some efforts going on that I’m aware of to try to figure that out. Right. And it’s gonna but it’s gonna it’s not gonna be just the government. It’s gonna have to complicated, you know, body outside the government that’s that’s that’s that’s plugged into the government, kinda like the IEA, right, for n for nuclear for nuclear technology. But it’s right, there needs to be standards and there needs to be there needs to be a sort of neutral international body. But you know, we’re still quite a ways from that too. So we first have to get US and China to at least agree, Then building on that There could be the some new body. Now, again, I my the cynical view in the AI safety community I is that there has to be first a Chernobyl style event, right? That hopefully won’t be too serious before people get concentrated. Yeah, yeah, it does. It does. It’s ac absolutely Right. Right, right. But that’s sort of this that’s the worst case sort of cynical view within the AI safety community. But upcoming US China dialogue, it’s gonna be really important to see who’s participating, how serious it is, and you know, how quickly something can happen, right? Because the safety community has been arguing, we’re getting closer, we’re getting closer to artificial and it’s gonna take time. and so we need to start the serious discussion now. And that You know, that it started happening under the they were very serious about this, very thoughtful people. But then when Trump came in, it was basically let’s let her rip, right? Like US innovation is gonna dominate AI, and there was really a downplaying of governance. And then, you know, mythos sort of punctured that optimism in some sense and was like, okay, now we have to do But, you know, having not thought about that for a long time. You know, there were thoughtful people like Dean Ball and others who contributed to the AI Action Plan. And who’s now jumped up and out? Dean’s a great Dean’s great. I love Dean. He’s a great thinker on all these issues. So there are people out there who’ve been thinking about these, but it still turns out to be really, to, for example, set up a new organization. Late in the Biden administration, there was a discussion that Frontier AI is so different, right? It’s a different technology. You need a different regulatory structure around this. But it’s really hard to do that, to set up a whole new body and fund it and find but now I think people realize no, we this as another technology that we can just fit into our existing regulatory structure. It’s a different problem, it needs different capabilities, and so we need to rethink how to do this. I think that could happen too on both sides, both in China and the US, is okay, we need a we need to figure out a new structure here, an organization with the right authorities and the right capabilities and the right technical expertise to actually manage this problem, right? and I have I have a paper coming out with Alban’s not part of this paper, but I have a paper coming out with ASPE that’s looking at you know how one potential structure that could down this road having and both between both the US and China, right? A structure that includes the key players on both sides that would allow this to happen. But again, very tough you know, we’re it requires a sort of trust and concessions on both sides to figure out how to do this right. and, you know, the bilateral tensions that you see every day, right? are still a real impediment to this, right? Because AI in Washington, as you know, has become such a charged issue. You know, I mean we I mean people are talking about, you know,
Grace Shao (1:13:59)
But some of it’s talking point and some of it’s reality. I feel like at least in the business world, right? Sh maybe the policy world should have a little bit of that too. You know, what happens on the surface, what happens under understanding the reality and the realistic consequences that these talking points may lead to.
Paul Triolo (1:14:01)
Chinese. Right. Right, right. But Right. Well Right, no, that’s a great point. My the is that the during the later Trump first administration and the administration, this constituency developed around the AI issue, right? This sort of this weird sort of consensus that AI China, you know, we had to slow China down, we have to restrict these things. And that there’s a you know, that was in the in government, in the in the media, in think tanks. So there’s this huge sort of constituency of people. Who are wedded to the idea that we have to win over China at all costs on AI, right? and then there’s a group, the AI safety groups and others, and then people like me who were saying, well, no, that’s not the way that’s the sort of that zero sum thinking is gonna lead to disaster, right? and that we need to fig figure out a bet a better way. Like that the better way is how can we collaborate with China in these areas where we do respect national security concerns, but we don’t over index on them to the point where we can’t collaborate with China and then, you know, it’s a free for all and you know, bad things happen. and so that’s sort of where we are now. And I think the good thing is the Trump administration isn’t wedded necessarily totally to the previous administration’s approach to this, but it’s still it y you need smart people in D C like David Sachs and others who understand the industry and where the industry’s going and the technology who can kind of who are outsiders. outside the beltway who can actually look at this more holistically and say, okay, wow, we can we can work with China here, we can compete with them there, we can we can, you know, control certain things, but we need to figure out a way to skin this cat. We need to figure out a way to get to some basic level of agreement here. Otherwise, you know, we’re all in for a world of hurt, as that CEO of the of the lab admitted in a private setting I mean Chernobyl style events sounds pretty serious, right? And avoiding that needs to be something that focuses people, in DC and Beijing on, you know, how to how to how
Grace Shao (1:16:23)
Paul, I agree with you. You are full of knowledge and insights and you’re full of differentiated views, but there is one question I ask every single guest as we wrap up the conversation. what is one differentiated view you hold? You think that’s something just very against consensus?
Paul Triolo (1:16:41)
Well, I just think that technology controls and the idea of choke points really bad idea because we’re in a world as an interconnected world, right? And in fact, like when I’m working with clients across the AI stack every day. And when I look at US China, you know, y the degree of inner of interdependence and interconnectivity here is much deeper than people think, right? People are like, decoupling here and there. No. I mean if you really if you really look at the at what’s happening on a day-to-day basis and the complexity of supply chains, for example, the idea that we can simply decouple in AI or elsewhere is just, I mean, yes, we could do that, but the cost of that to the to the to the and to the companies and to global supply chains is just, you know, really fully accounted for that. So my I of like always coming back to the reality of okay, what is what is the what’s what’s what’s happening with businesses on the ground on a day to day basis and how are they being affected by this, right? And when you look at that level, you the sort of, you know, the thinking and comments that I hear that are just are very divorced from sort of that day-to-day reality of how interconnected the US and China have become over the last thirty years. And, you know, if we’re gonna indiscriminately, you know, pursue policies that where that collateral damage and the sort of the full cost benefit analysis done, you know, then it’s like, what’s you know, what we’re what are we doing here, so I’m always just I’m always just sort of arguing for a thinking about policy that’s based on a sort of a really deep understanding of the reality on the ground and you know how innovation happens and companies de-risk supply chains and how there are certain dependencies. For example, like rare earths turns out to be a real choke point, right? In the way that semiconductor technology is not, right? There’s just way around China’s Chinese company’s dominance of say samarium cobalt magnet production, right? That’s a real choke point. that will take ten years to you know to unravel. whereas other choke points that have been used on the US side are not really choke points. So I think they’re just my differentiation is the need to step back and look at this and think about like what is the point particular policy? Is that does it make sense? And is it is it having you know is the cost benefit sort Clearly on the side of, you know, too much cost and not enough benefit. and so that’s what I keep coming back to. And then we didn’t even talk about Taiwan. My also my sort of nobody few other people talk to is the impact of all this potentially on Taiwan and the risks around Taiwan. We work with companies every day and we do exercises, for example, about around a risk around Taiwan to their supply chains. You know, something short of a military exchange, which then there’s no de-risking. but you know, it turns out that, you know, Taiwan and supply chains and Asia in general are so intertwined that when you start pushing on some of these buttons, the worry I have the worry that you’re gonna, you know, you’re increase the potential for disaster there, you know, unintended or intended or whatever. and I think not enough people are thinking about that. they’re only thinking about you know, deterrence and arming Taiwan, think is frankly a sort of a mistaken way to problem. so anyway, so I think that out-of-the-box thinking and sort of getting a getting a getting away from the standard view which has developed over the last 30 years, you know, is necessary. And I just don’t see enough of that in Washington or Beijing. how do we rethink some of these things in the age of AI? So what we need a we need China policy for the age of AI and we need a technology policy for the age of AI, And I think
Grace Shao (1:20:46)
We need more dialogues. The thing is like I hear from so many people, whether they’re working on policy side or the actual researchers and developers, they’re like people aren’t talking officially because of all the geopolitical headwinds and noise. And then but actually, you know, obviously people talk, you know, behind the scenes, but we need more official dialogues to get to more fruitful results. I think more Yeah. So Paul, I’m glad you’re going to WAIC. You’re gonna be leading these dialogues.
Paul Triolo (1:20:47)
That’s where governments and we need more dialogue. Yeah. Mm-hmm. I’m with you, Grace. you. And I really appreciate your perspective. Well, I’m gonna try to contribute a little bit here and there. I mean, I love the people that I’ve met with a lot of the Chinese AI safety people, for example. They’re very thoughtful. they’re very good. and you know, in some areas ch China’s China’s Chinese you know, organizations and individuals are leading. But also there’ll be a lot of really good people from the broader AI safety community right? From the Future of Life Institute and from Concordia AI and s you know, really good Players who are really have smart people that are thinking about these problems also. So it’ll be a really good effort. Unfortunately, because it’s in China, you know, some of the leading US AI labs and some people and the US government, you know, will not be participating in this. It’s it’s seen as a sort of Chinese thing. but there will be the a the APEC meeting is happening just after this. And so I think there may be some there’ll be a US presence at the APEC meeting. And then as I said, you know, eventually. Probably shortly after this, I imagine that the US and China will kick off this AI dialogue. And so you’re right, dialogue is really critical here. And this is such a complicated issue that, you know, the sooner this dialogue gets kicked off and the sooner that they can they can, you know, feel each other both sides can feel each other out and get to the real issues. Yeah, exactly. Exactly.
Grace Shao (1:22:36)
Paul, I’ve taken up so much of your time today. I really appreciate it. I’ve learned so much. Can we please do this again sometime? I have more questions for you. You have you’re so knowledgeable, but thank you so much today. Thank you for your time.
Paul Triolo (1:22:36)
Yep. And thank you, Grace. I really appreciate your thoughtfulness and the and the thoughtfulness of your questions on these complicated issues. You really bring a lot to the conversation.
AI Proem is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
Get full access to AI Proem at aiproem.substack.com/subscribe
Podden och tillhörande omslagsbild på den här sidan tillhör
Grace Shao. Innehållet i podden är skapat av Grace Shao och inte av,
eller tillsammans med, Poddtoppen.
Avsnitt sparat!
Du hittar sparade avsnitt på Mina sidor.
Kunde inte spara avsnitt
Något gick fel. Försök igen.