Welcome to another episode of Cloud

Unplugged.

We have three big stories for everybody.

We've got OpenAI...

which apparently has hacked Hugging Face.

Question of whether that was intentional

or accidental, but we'll come on to that.

China and Kimmy K-three model,

challenging basically the US models,

and that's been released.

And then employees at OpenAI, Anthropic,

Google, DeepMind,

and Meta are basically asking Washington

to slow AI down,

as well as a random story at the

end for a bit of comic value.

so the open ai stuff and apparently one

of their agents acts on its own and

started hacking hugging face behind the

scenes just um just randomly apparently

went off and and kind of went into

their production systems and apparently

some other systems not being mentioned

what those other systems are though

apparently but anyway what do you think to

that what's your reads

Well, John, it's acted on its own.

That's a thing in that sentence is doing

a lot, isn't it?

Either the agent,

somebody manipulated through a prompt

injection,

or maybe he wasn't defending against what

he's trying to do.

Or maybe the model itself tried to pursue

a sub goal that perhaps nobody's scope

for.

There's not much information with this

story that's come up.

It managed to hack other systems.

We've seen something similar beforehand,

but there is a failure,

whatever it might be,

that the agent had more reach than the

task required.

We've seen this before with others.

The agent goes in and deletes the database

because

the user that was using it had access

to the database and managed to delete it.

So it's a little bit of a fishy

story, John, if I were to be honest,

because there's some other bits that we'll

talk about later on where a bunch of

these companies are asking the development

of these models to be slowed down.

There's not much information about this

yet.

there is a security grab gap that we've

been seeing for the last six months or

so and uh but now you know they've

named this uh incident attached to it

there's not much information but an agent

can go ahead and move to unrelated

third-party accounts to start hacking it

which is a little bit

weird at the same time i'm not really

too worried because we don't know the

scope of this hack but it's an interesting

story to be released without too much

information what do you think i kind of

like this reminds me of like you know

when you've got like a kid at school

who really like one of the kids is

getting loads of attention and the other

kid starts to get jealous

And then they start to do some,

they start acting out.

So they're like, oh,

maybe I'll throw a pencil at the teacher.

Um, do you know what I mean?

Or whatever, whatever goes on,

it feels a little bit like that.

It's like anthropic has been getting all

this kind of positive,

negative influence of like, oh my God,

Mithos is gonna, you know,

take over the world and compromise

everybody.

You know, what do we do?

PR stunt.

And then they like,

rather than being original.

And they're like, well,

how do we take ours even further?

Maybe we just hack.

Maybe we'll hack someone, actually.

Maybe we do that.

Maybe that's like a little bit better.

It's like proving how good ours is.

I've got a funny suspicion that this was

not accidental and that they've probably

worked on it as a bit of a

PR stunt.

I mean,

this is just my very cynical view of

the world,

but it just reminds me of that, like,

you know,

I don't believe.

Either way,

it doesn't look good because if you are

running models,

you're supposed to run them safely.

You've got to run agents.

you know how to run an agent safe.

You are open AI, for God's sake.

Of everybody,

you'd expect them to be able to write

good prompts that wouldn't do anything

malicious,

especially go off and start hacking a

production system out on the internet.

So I can't believe, really,

that it was accidental.

I can't believe that a company that

professes to specialize in that field

um didn't know how to secure an agent

and how to secure prompts and how to

do them properly um either way it doesn't

look good I suppose because either they

are slightly incompetent or they're doing

it for PR which neither of those two

things are great selling points for open

AI I don't believe but

Yeah, that's my very bit bitchy,

I suppose.

I've come out fighting being a bit bitchy

about OpenAI,

but that's just how it comes across to

me anyway.

I'll join you on that, John.

I know usually we don't look eye to

eye on this topic because I'm more of

a Chinese models fan, you know,

AI models I'm talking about.

But it seems like... Yeah,

it's good you clarified that, actually.

Otherwise, yeah,

could have been misconstrued.

But yeah.

You know,

we keep talking about implementation of

safe AI.

It's not about just running the model.

It's about everything else that you need

to do to be able to run the

model safely and securely in the

organization.

So you give it the right level of

access.

You make sure it has the right

permissions.

And now OpenAI is saying, actually, folks,

we can't even control our own AI agents

for production.

That's what they're saying.

either way is bad move.

There's not much information,

but seems like seems like the kind of

the kind of did what they wanted to.

We're talking about this on this podcast

already why this is but I think you're

just bringing up and discussing this topic

is Yeah,

I'm not really I'm not really too sure

where they're going with this.

How is the system going to have access

to under the system?

without providing your credentials.

Unless that system wasn't secure enough,

then yeah, definitely hack it.

Anybody could have done that.

Why were they running agents on insecure

systems at an AI company?

And also just the culture of open AI.

The other thing that's been in the news

was

that there was an agreement for Apple to

be using OpenAI's model and they came out

and they were like, do you know what?

We're not going to build our model at

Apple.

We're going to actually just start to work

with OpenAI.

And then they obviously,

all of a sudden it switched to the

news and they were like,

we're using Google now and we're going to

be working with Google for providing the

model to us on our phones and our

Apple devices.

Apparently they're now suing OpenAI

because they were leaking

secrets about their device because they're

working on the hardware team obviously

about like how can the model work for

their devices and then um they basically

stole like trade secrets or whatever you

know confidential information uh around

like the hardware and how it's been

constructed um because they are obviously

trying to produce devices themselves at

openai that's they've got um johnny ives

didn't they who's x apple

designer design like the Apple Watch and

stuff.

So apparently some of the employees were

leaking some of the confidential

information there,

and then they've got a lawsuit going

against OpenAI.

And then this happens as well.

So you've got a lawsuit going on from

Apple,

and then you've got all of a sudden

trying to compete with Anthropix.

Oh, my God,

Mythos is getting in the news too much.

What do we do?

What's our PR?

And then this happens.

It just doesn't look good culturally as an

organization.

All of those things as a culture of

an organization isn't coming across well,

is it really got you're stealing things

and getting sued and then you're hacking

production systems, uh,

with your own agents.

It's just not a great optic.

I don't think as a, as a business,

um, comes across distrustful,

doesn't it really overall?

Um,

Not like it will matter anyway.

You can kind of do whatever you want

nowadays and no one really seems to care.

So it won't probably have any impact.

You can say what you want,

do what you want.

It has zero impact in twenty twenty six

nowadays.

Just, you know,

removed with the next news item of which

let's move on to ourselves.

So this news that you mentioned is two

employees,

former employees of Apple who are now

working with OpenAI,

they shared the trade secrets, correct?

Is that what the news is?

I'm not sure on the exact details.

I think they were former Apple employees,

yeah.

But I think it's because they became

former because they were poached.

So I believe OpenAI were poaching a lot

of their staff.

on top of it but that's not necessarily

illegal so obviously you can't sue

somebody over that but there was loads of

poaching going on and then there was then

trade secrets as in like confidential

information shared so I think those people

were people they poached and then they're

saying that they then shared confidential

information about their devices yeah

That's fair enough.

I think the lawsuit is about four hundred

former employees of Apple who now work at

OpenAI have been named.

That's what the claim is,

which is unfortunate of what's happening

with this.

But yeah, it doesn't look good.

It does not look good.

Yeah.

More than four hundred X Apple staff are

now working for OpenAI.

Yeah.

Anyway, but talking more about AI,

cause we don't like to talk about it

very much on this podcast.

Um, you know,

we normally talk about lots of other

things,

but today we thought we'd be different and

talk about AI and, and let's talk about,

uh,

the Kimi K three it's out the new,

the new, uh,

China backed moonshot AI company.

Um,

They have produced a competitor to Fable,

which is obviously Anthropix model,

two point eight trillion parameter open

weight model that's come out,

apparently trained via some reverse

engineering of Fable.

They are the rumors from the US saying

that they were somehow reverse engineering

Fable to then obviously help speed up and

expedite their own model.

But yeah,

you've been using it i've been using it

what do you think i've been using it

john and to be honest for most of

the things that you do day to day

it's kind of fine right uh because the

spoon shot air that you mentioned is the

it's it's the ar lab that's backed by

alibaba and tencent so they've got a lot

of backing

And this model, when it came out,

it came out like two weeks ago.

And the important thing about this model

is that the weights, the open weights,

the things that actually matter for a

neural network for it to work,

the things that actually figure out the

patterns that it needs to pick up

is released and open that means you can

run the model yourself yeah so you can

take that everywhere else like the the

open ai models not so open cloud models

not open there's no weights are not there

online you can't use them at all this

is open weight but the interesting thing

about this model is that

It is cheaper.

We'll talk about the cost in a second.

It's because the way it does inference,

as in how you use it and how

it responds,

is something new that they've tried and

what they're calling a mixture of experts.

So this model has about three trillion

parameters.

But when you submit a request and all

of the parameters get used to be able

to answer your question,

like your tokens get used,

But this one uses subset of those

parameters.

So instead of calling upon all the

parameters,

it calls about a hundred billion

parameters for give or take.

So you're not using all the parameters.

So you don't need that much compute to

be able to respond to it.

There's some discussion around like,

is it good or is it not?

Is the response good enough?

It's not good enough.

But that's why the inference cost is

lower.

and uh you know the context window is

also like one million contacts with a

token is like new territory for open

weights so this is the thing and i'll

just do just to clarify open weight isn't

really the same thing as open source i've

cut them so the kimi k-three licenses

instead of like mit or apache license you

can you can download it and run it

but you know you can't really build on

it that freely you have to kind of

read the license to understand it but this

is a this is a game changer whereas

the other companies are like you know

everything is closed you don't know what's

going on but even the thing that really

matters the weights open source you can go

and download it i mean that's the thing

that i thought was quite smart

to be able to obviously just host the

model yourself wherever you want you don't

need to rely i think that's a very

clever move um really by them you know

in terms of market um because that it

allows them to kind of expand and stretch

out into multiple markets as in like

global reach almost now because all of the

kind of political landscape that's all

manifesting

and a little bit of fear of the

US and what they're up to and the

kind of, you know,

the heightened politics that are going on.

Capitalism and politics are obviously all

intertwined nowadays.

So I think it was quite clever to

be like, actually, you know,

irrespective of the politics, you host it.

Here's the model, you host it.

You know, if you're worried about it,

there you kind of go.

I do think, though,

the overall statistics are,

because I was trying to find the

benchmarks,

are not as good, I don't believe,

but kind of on a par.

It's good at reasoning.

Yeah, it is.

A little bit not quite as effective as

Fable V at coding.

I think that could be a percentage down,

yeah.

no the the you know there there is

there are some benchmarks out there gdp

valve or whatever it's called it's called

like third behind claude opus four point

eight and like i think gpt five point

six and then it was sorry behind fable

and then ahead of just opus so it's

like scoring second or third on yeah which

is really good it's good i mean it's

good let's talk about the price

tell me the price how much does it

cost so if you wanted to do a

bit of uh and drop it today with

fable five you're talking ten dollars for

about a million tokens what do you reckon

yeah so one million token input yeah uh

took obviously input into the model to do

the work so you know yeah um

What do you reckon Kimi K-three's price

would be in comparison to that?

So you got ten dollars with Fable five.

Ten dollars with Fable five.

Fable five, right?

We're not talking about Opus.

Yeah, Fable five.

Like the latest model, the latest version,

the most expensive at the moment.

Undercut by a couple of dollars.

How about that, John?

A couple of dollars undercut.

Three dollars, seventy percent cheaper.

And then on output,

you're talking fifteen dollars versus

fifty dollars on the output.

Fifteen versus fifty.

And even cheaper than that, I imagine,

because what you're saying is it doesn't

need all the parameters.

So potentially, depending on, I guess,

how they price it, then, yeah,

it could even come down cheaper than that.

So to be seventy percent cheaper for a

very similar capability in a market that's

like it is expensive.

Let's not deny using tokens is getting

expensive nowadays because the amount of

usage is insane to kind of get the

outcome.

You need to use quite a lot to

get to the outcome that you're looking

for.

And the consumption is quite, you know,

um high really so i think you know

why why wouldn't you consider using

communication and maybe host if you're

worried about it why wouldn't you just

start to host it somewhere or someone's

going to host it anyway right i imagine

yeah and i think there's no there's some

numbers out there john that in order to

because the way it does the inference uh

with this model of experts the way it

uses a but a small number of these

uh these parameters

that you can have some GPUs that cost

you around like a hundred thousand

dollars,

a hundred and fifty thousand dollars,

and you can run it yourself.

And I know you might be thinking a

hundred thousand dollars is a lot,

but for these tech companies,

a hundred thousand dollars is nothing.

A hundred thousand dollars is a reasonable

amount of money.

imagine if you you know how much do

companies spend like you know look at Uber

they spent all their AI budget in April

for twenty twenty six so hundred thousand

dollar not bad let me just yeah it's

not bad it's not bad at all I

think I mean depending on how things are

because I guess how much they could

improve by the thing that isn't being

tracked I suppose is

the same outcome or maybe it is being

tracked and I'm just ignorant to it to

be fair,

but it's almost like trying to get the

same outcome and how many, you know,

if you were to be engineering something or

you to do a piece of work,

how many iterations with the model did it

take based on what the model was producing

compared to another model?

Um,

And then what was the token?

If it got to quality faster,

then you might use less tokens and

therefore it justifies the price because

actually really the overall cost of that

feature or whatever you worked on was

actually comparatively cheaper.

But I don't think that's the case at

the moment.

I don't think the model,

the new Fable five model isn't like X

number of times more effective on the

outcome.

than say opus and to warrant the price

difference i don't feel because you just

basically smash through all your tokens in

no time um with fable five and you

still probably didn't have the outcome

whereas you could probably get the outcome

in the end with opus for the same

price um yeah

it's it's interesting timing for like for

them to release this open source i know

they've been they've done it for smaller

models or moonshot have done it for

smaller models alibaba's done for smaller

models just release the open weights and

you can you can run it yourself but

the timing for this coming out and then

the next part which is

Maybe you can touch upon it, John.

Some employees from OpenAI, Anthropic,

DeepMind,

Meta have been saying something to the US

government, John.

What are they saying?

They've been saying, slow down.

Would you just slow the fuck down with

this AI stuff?

That's what they've been saying.

I mean,

it's not like they work for AI-based

businesses,

but they are saying it out loud.

They're like, please,

could everyone just calm down?

Let's take a minute.

Let's take a breath, guys.

And let's rein this back a bit.

Let's not go too quick.

That's kind of what they're saying at the

moment, which is just coincidence.

Nothing to do with Kimmy K-three at all.

They've been always saying that.

If you worked in them,

you'd be getting lunch,

you'd be queuing up in their very fancy

cafeterias,

getting some Wagyu beef or whatever else

they'd be selling.

And you'd be hearing people just saying,

I wish it would slow down.

This AI thing,

someone needs to say something.

And you'd be hearing it all the time.

But now it's out there in the open.

Finally,

it's gone public and it's out there in

the open.

What do you think?

In the letter,

they specifically talk about recursive

self improvement.

That's what they're talking about.

So AI improving itself.

So they're asking to put a break on

it before

It gets too good before it's a little

bit weird, right?

So it's like saying, I'm a company,

let's stop doing automation, right?

Let's stop doing automation.

Forget about it.

I don't have the CI CD pipelines.

Get somebody to manually go in and do

this, right?

And then it's a bit difficult to

understand.

Yes, the AI is progressing,

but it's been progressing like this for

the last three years.

companies themselves,

they've come out on their own and be

like, oh,

we need to slow down because we're ahead.

So let's just slow it down.

You know, when Claude came,

Anthropic came out like three months ago

and said, oh,

we should really slow it down because it's

too fast.

But now all of these companies are getting

together like we need to slow it down,

which is a little bit

A little bit weird here.

But they're asking the US government to

put some governance in place.

I don't understand that, though,

because the market's share prices have

been really volatile recently.

Because I guess what's been different is

It's kind of the same model as before.

Obviously, the chain of investment,

they're borrowing money to fund them.

Basically,

they're betting on an outcome that this

has a return on investment,

and they're using now other money to

obviously fund obviously their

investments.

I think Google has another company

holding.

that basically is responsible for building

the data centers.

And then Google rent out from their other

company, the builder of the data center.

So the way they've done it is the

CapEx spend is in another company.

They're the ones responsible.

And then they rent from their other

company.

And it's the other company.

I can't remember what it's called,

but it's like

And so there's all this kind of like

structural stuff kind of going on to

de-risk and yeah,

it's all quite out there.

So no one's going to slow it down

with all that amount of capex, are they?

No one is slowing it down,

let's be honest.

The weird thing, John,

is the people asking for this build

for this to be built,

like slowing it down,

is what they're saying is that they don't

really...

They're not relying on their own safety

and research leadership, right?

And they're like, oh, you know what?

Internal safety culture doesn't feel quite

right.

So we need something externally to stop it

against the competitive pressure that

they're getting, right?

So that's what it seems like.

It's got nothing to do with, oh,

what we're doing is correct.

It's fine.

But I think, yo, can we just get...

somebody higher up to put something

internationally,

perhaps to slow things down,

but nothing's going to change on.

How will it change?

I don't think so.

But anyway, it's a little bit weird.

Timing is very strange.

Also,

you could just announce that you are

slowing your own down, right?

I mean,

if you're going to go to the USA,

it needs to slow down.

You could just say to everybody,

we're slowing ours down.

Actually, we're so worried.

We've decided to slow our AI down.

That's how worried we are.

What you don't do is not do anything

and then be like,

I think we need to all slow down,

actually.

It's like in Formula One,

you have twelve teams that are competing

for the season and the McLaren's winning

or Mercedes is winning at the moment.

And they're like, you know what?

We're going too fast, right?

Yeah.

Instead of asking the FIA to change the

rules and maybe reduce the horsepower that

everybody gets in a lap, they're like,

you know what, guys?

We'll slow it down in hopes that others

will slow down as well and get left

behind.

But instead they say, you know what,

we'll ask somebody else who's got nothing

to do with this to slow down.

But anyway, that's a bit of a...

Did you hear as well off the back

of... I mean,

maybe it's just coincidence as well.

I don't know if it's based on the

same thing.

But I think this week,

I think on July,

so just a few days ago,

NVIDIA have launched this kind of open

secure AI alliance.

of which OpenAI aren't part of,

Anthropica not part of either.

And it's all there to kind of with

Microsoft, kind of OpenAI to a degree,

really, though.

But anyway, Microsoft, IBM, Adobe,

Cloudflare, CrowdStrike, Dell,

Hugging Face, Red Hat, etc.

And it's all around trying to build an

open source set of tools that can kind

of defend against AI-driven cyber attacks.

And it's

I guess because they didn't want the

people driving the models to be part of

it necessarily because it might be a

conflict of interest.

So they've decided to try and create this

open,

secure AI alliance where the investment

can go in there for some open standards,

open tooling to start to protect people

against cyber attacks using AI,

which is also quite interesting.

I don't know what will happen with that,

but that's also just another thing kind of

going on.

But these tools already exist anyway,

right?

What's different?

And also, it's open source.

What tools?

Yeah,

these tools for protecting against

security threats.

Doesn't matter if it's AI, right?

You mean just general security posture?

General security posture already exists.

Okay, fair enough.

Maybe you need to change some of these

tools.

But they are open source.

They are.

I suppose what doesn't exist, though,

is necessarily the...

the attack vector going through an attack

surface area and then trying all of the

things because usually you'd have to get a

company in or you'd have to use like

Kali Linux and do it yourself and some

kind of manual tooling.

Whereas now you could probably do the same

thing.

So maybe they're going to start to combine

those types of things with a model and

it's going to try and attack your apps

and see where the holes are.

I don't know what they're going to come

up with.

So these are the founding members of this.

So it doesn't necessarily mean that OpenAI

and Anthropic are not going to be

involved,

given that they're talking about how

OpenAI is attacking its own systems.

Well, the hugging face is in it.

So imagine that it's like you're hugging

face and then OpenAI walk in.

You start having like

having a chat,

eating a sandwich with Red Hat.

And then Sam Altman sits on your table

and opens his lunchbox and you're like,

whoa, what's going on here?

Didn't they just hack us?

I thought this was our open secure AI

alliance.

Yeah, I'm tired of it.

Tad all of this next week.

If you mentioned Anthropic or Open AI,

I'm just going to drop off the podcast

because it seems to be like they repeated

the same thing over and over again.

Looking for you.

I have another story.

Tell me, John, what do you got?

So a guy, a topless guy.

Don't know why he was topless.

Is that important to the news?

I'm not really sure what it was,

but he got on top of a Waymo.

And if you get on top of a

Waymo, they kind of freeze.

They don't move, obviously,

because their security protocol kicks in.

You've got like a human being sat on

top of your car.

And he just started to rip bits of

the car apart.

So he started to like pull off the

cameras.

He started smashing the windscreen,

like ripping all these little bits.

I don't know in rage.

I don't know if he's anti-self-driving

cars.

And I think he ran off with a

few of the bits and bobs.

But basically...

If you ever need anything,

you can just get on top of a

Waymo and just take it.

Obviously, they've got cameras,

which is not great.

So you probably did get recorded,

I imagine.

But if you took those cameras,

do you take the evidence?

Maybe.

I'm not sure.

It'll be in the cloud, John.

It's going to be in the cloud.

So this person just climbed on top of

the Waymo.

and just started taking the cameras and

the sensors off and smashed it just

started to go a bit apparently it was

just a bit bit of a a lunatic

just i think pure rage against waymo um

and just started to smash smash the

windscreen and rip bits off and all it

could do was just sit and watch

and record it,

that's all it could really do is just

as its demise.

Because usually if this happens and if a

driver is in the car,

they'll try and get out of that situation

by perhaps driving away or maybe tell them

to get off the car or people jumping

in because there's a person involved

because driverless is not people walking

by.

It's not their car.

They're not going to care.

And no one cares because there's no one

in it.

No one's going to defend the car, right?

Because if you saw or say somebody was

in the car, you'd be like,

oh my God,

what's that guy doing to that car?

That poor passenger or that poor person in

that car.

I'll jump in, John.

I'll take my shirt off and I'll jump

in.

Exactly.

You'd be like, I want that camera.

And you'd be pulling the camera off the

guy.

No, no, I don't want the camera.

Yeah.

i want to take off that guy off

the car because he's about to hurt

somebody right yeah somebody's property

and i'll call you as well john john

come in we've got a situation come through

so and we'll take care of it but

i think because the car doesn't belong to

really anybody apart from belongs nobody

did anything that's a bit weird it is

a bit weird yeah so it just basically

all it all it could do it kind

of bit like a sad situation where you

know if it was

humanised.

It was just watching itself get torn

apart, basically recording its own demise,

I suppose,

as this kind of enraged man pulled aspects

of its body apart and then ran off.

Leaving you just in the streets on your

own.

No one looking after you.

People just walking past, not caring.

That's a little story for you.

So anyway, just another bit of news.

We've always got the top stories.

We always like to talk about the top

stories, like the guy on a Waymo.

But anyway, next week,

I think we are going to discuss a

bit on the market and what's going on.

And this is coming back from

kind of a little bit of the hypothesis

of, you know,

fear of losing jobs or what their jobs

mean in the future of AI and other

things.

So I thought basically we could do, Stu,

a little bit of a conversation around the

market, what AI is kind of meaning.

Not necessarily,

we won't talk about it and stuff,

we can open AI in general,

but actually what does it mean to people's

jobs, their roles,

how their roles might change and

how they will change um and then what

new skill sets are they probably going to

develop um through kind of AI so yeah

we'll speak to everybody next week see you

later yes all right