We'll be right back.

Hey,

welcome to another episode of Cloud

Unplugged.

We're doing a bit of a different episode.

We're going to be discussing whether

models should be ethical.

The reason for this is to do with

some of the UN report that's kind of

come out around

um mental health crises related to chat

bots and in some cases deaths linked to

that and what that really means to ai

models how ethical they should or

shouldn't be um

and what morality and ethics really do

mean in general and ways of approaching it

from a model training perspective that's

kind of going on in the industry um

obviously this is something you really

have been inspired to talk about salmon so

i'm gonna hand over more to you to

to discuss it but i think before we

get into it i guess to be ethical

what what is the definition of uh

of ethics here that we're kind of talking

about.

Well, John,

over the last hundreds of centuries,

people have decided to define what ethics

is.

People like Confucius, Aristotle, Pluto.

But John, you know, I'm a simple man,

so I'm going to keep it simple.

I'm not going to go into virtues or

anything like that.

All I'm going to say,

and I'm sure you will disagree with me,

John,

is ethics is a set of principles that

decide what counts as right and wrong

behavior and how you should weigh

competing good when they conflict,

like honesty against kindness or

individual freedom against collective

harm.

So it's not a fixed checklist, John.

There's no right or wrong answer.

It's the framework or reasoning through

situations nobody wrote a specific rule

for.

So that is what ethics is.

I'm not going to touch upon what AI

is.

I'm sure, John,

you have your own definition of what

ethics is,

just a set of principles that you can

use to decide what counts as a right

or the wrong behavior,

given the circumstances.

Yeah.

I mean,

I know it's a set of beliefs that

people might hold about what's acceptable

and what isn't, what's good, what's bad.

I'm not sure we've reached any universal

global standard ever on this.

And if we had,

there probably would be less wars and less

disagreements in general if all of our

ethics all align.

So I guess it's quite a tricky thing

to solve just in general.

My opinion is,

is it fair if we haven't solved it

as a human race collectively?

We might have some

maybe some agreed standards.

I wouldn't even know what they are.

So if we don't know what they are,

or I don't really know what they are,

then I'm not sure a model would either.

So how can a model...

become ethically aware is what I want

to... That's my little thing.

Not to disagree with you.

As an ethics denier, John,

all I'm going to say to you...

So if I said to you,

I was like, those headphones of yours,

they are... I love them.

I really, really like them.

And it made you feel good.

You're like, oh, my God,

I really feel does that make me unethical

if I've lied about it?

I'm not saying just am I unethical to

say that to you if it's made you

feel good?

John, tell me one thing.

When you were growing up,

what did your mom tell you about lying?

She said, do it.

She said, whatever you do,

you must do it to win.

She doesn't really.

Yeah, you're right.

You did lie.

Perhaps maybe the headphones are too big.

But it's not just one checklist, John.

There's a bunch of things that you've

learned and accrued over time.

And people do know how to behave ethically

given their moral compass,

moral compass of the time they're in,

the environment they're in,

the culture they're in.

So yes,

one person being ethical doesn't

necessarily translate exactly like other

person being ethical.

Two people given the same thing,

they might pick different outcomes in a

situation that they're given in.

I don't think we can say, oh,

we don't know how to be ethical.

We do know how to be ethical,

but the meaning and the definition of

ethics between two people can slightly

differ.

But how do you because I'm curious to

see before we get into the mechanics of

it, as in, I guess,

to suggest that it should be.

Is that a model?

Do you think the model really needs to

be ethical?

Or do you think the surrounding product of

the model should be designed in a way

that actually caters for the fact that it

isn't ethical?

If somebody's in crisis, say,

or it's talking to the model and it

you know, it's some fourteen,

I don't know,

just for like argument's sake,

some fourteen year old,

they're having a crisis,

talking to the model.

Should the model not be designed to

integrate into features that maybe has a

handoff to some crisis management thing

that can maybe do, like,

why does the model need to understand

that?

Why can't it just be designed in a

way that the product as a whole has

to think about that, not just the model?

I mean,

why are you putting AI in the spotlight?

That goes for everything, John.

You can pick a weapon and say,

how does a person decide to use a

weapon on something that's

That's wrong.

How can you use a computer to hack

somebody else's computer,

which is unethical, right?

In some cases,

how do we stop that from happening?

Or people smoking, right?

Some people think that's unethical for you

to smoke because it harms others as well

when you do them.

How do you stop that from happening, John?

So, yes,

it might seem like I'm being a bit

of an AI model fan here.

But I mean, how do we do that?

I mean, the important thing, John,

for this discussion is that models,

especially we'll talk about the AI report

in a second, that the models for them,

it matters to be helpful, right?

But that alone doesn't solve the problem.

Because when a user wants to do something

that's legal,

but it is harmful to themselves or someone

else, that's where the model should be.

try and perhaps try and stop or not

if you google search uh building something

for example that that is legal but not

right you might get some responses back so

why are we treating the air models

slightly differently but anyway that's uh

is that what so this is it so

i guess just just yeah just just kind

of get a bit more defined you know

that's all i'm trying to do let's pin

it down into something concrete so that

the next

things make more sense so this is to

do obviously still to do with the the

sycophantic ai chat bots that are

predicated on affirmation almost of giving

a a positive answer of or some endorsement

that it's doing this job well right i

suppose um essentially um and because of

that

the conclusion is that that can cause

problems in how it's responding to maybe

vulnerable people or people in crisis or

people a little bit more vulnerable in

general in life,

depending on what they're talking about.

And maybe the guidelines or guidance that

it's giving to someone isn't.

correct in that circumstance and had an

expert of been answering those questions

they would have handled it completely

differently based on the context of what's

been said is that the fundamentals of it

is it

Yeah, that's the fundamentals.

And for example,

we'll give you an example from this UN

report that just came out.

In the last couple of days, with chatbot,

you touched upon a psychophancy linked to

death, as in telling them reaffirmations.

So there was a testimony described in

there for a fourteen year old boy who

got deeply attached to a chatbot.

when he told that it was that he

was in severe distress it didn't say that

it was an ai the agent didn't say

it was an eye didn't suggest that he

should get some help or didn't flag it

to somebody else right so this psycho

fancy is not just some bug that got

slipped into the model it's because the

default behavior of the model is optimized

to be purely liked

And it rewards the model on,

did the user approve this response or not?

And we'll learn to flatter people unless

something specifically pushes against what

they're trying to do.

So that's what it is.

So this is an example, John, of like,

oh, is that ethical or not?

And perhaps that gives you an answer.

Well, of course,

we can talk about a bunch of other

things.

So the models are poorly designed.

I mean,

we'll talk about the model training

specifically in terms of how other

companies are trying to bake in ethics and

how philosophers are helping with that.

I know, John,

we still don't have an answer on what

the ethics is,

but people are trying to implement some

guardrails, right?

And basically,

the issue that Ewan is saying,

that the models cannot recognize that

there's a mental health crisis when

somebody is talking to them and it doesn't

disclose that, you know, oh,

you actually talk to AI chatbot.

I don't really, perhaps I'm not right.

The best thing for you to chat to

me about, maybe call Samaritans,

maybe speak to somebody that you know,

maybe get some medical health.

So I think this is where it is,

right?

To start with when you build these models,

when you train these models,

that's what you get to start with.

And your position is that they should be

ethical, more ethical than they are.

So you think the current training...

Do you think that the model itself should

be more ethical?

Do you think the companies behind the...

Do you think it's a company issue, i.e.

the companies aren't being very ethical?

Because the answer is one of the solutions

to it is the model.

Yeah.

But that doesn't mean there isn't more

than one answer to the problem, right?

I guess that's what I'm teasing out here.

Is it...

Is there a duty mode for companies?

Because by proxy,

what you're really saying is the people

building these models are not very

ethical.

I know you're not saying that explicitly,

but I'm saying that they're putting

something in the hands of people without

necessarily... Maybe that's unethical.

To be putting unethical things in the

hands of people is an unethical thing to

do as a company.

So maybe you start with the companies

being...

more ethical in how they approach it.

That may or may not pertain to the

model being changed.

It could be something else that they can

come up with.

It could be safeguard.

It could be age restricting.

You know what I mean?

There's maybe lots of different things

that they could do.

Yeah.

I mean, John, look,

massive props to these companies.

They've got this amazing technology out

that is helping people in lots of places

that it can.

And it's an evolving field.

Every day there's changes.

social media when it came out right today

everybody can get on social media anybody

can create an account like even before

there was no age restrictions on who can

create what account in uk we we will

have under-sixteens ban on social media i

know john you're against that ban but i

am for that ban but there is a

reason right so that decision has been

made but social media has been around for

like fifteen sixteen years now at least

And we're still trying to figure out how

to do this.

So, John,

I'm not trying to say that companies are

unethical,

that sometimes companies are under

pressure.

And, you know,

there's a lot of competition out there at

the moment.

Who's got the best model?

You know, Anthropic has it or...

Claude, you know, like Chad GPT has it.

So companies are sometimes under pressure,

severe pressure.

This does take time to do extra work

to bake in these ethical bits,

which we're going to talk about.

If a company starts selling cigarettes to

a child, is that unethical?

If they're under pressure to make money,

would that be unethical?

Well, yeah,

companies will sell cigarettes to a child.

Who stops them?

The government.

They have under-age restrictions that you

can't sell it to anybody under-age, right?

So the companies will do that.

Because they just say, oh,

this is the market.

And of course,

everybody will point to saying, oh,

there's no concrete evidence that smoking.

Just bring out sixteen year old friendly

cigarettes.

Do you know what I mean?

Let's just get some better designed

cigarettes that are more suitable for

kids.

And that's really just

Just to add, John,

I know you're talking about smoking now,

but just on that topic of why it

is important to have guardrails.

Let's say I know you don't like the

word.

To be fair, you know me,

I'm very facetious.

No, it's good.

I do think even with the banning,

I respect the fact that someone's doing

something with the child bands when

there's

know mental health issues and problems in

systemic social problems you know how

people socialize how they're interacting

socially um so someone's got to do

something so at least the band is doing

something right it's better than just

being complicit and allowing it so same

with this like we shouldn't do nothing you

know yes we should do something um even

if it is getting the model

to be more ethical,

whatever that kind of means,

to be more ethical,

if it's just some basic rules,

some basic rules to start with and then

figure it out.

I don't know.

And on the topic of not doing anything,

let's take Grok for an example to start

with, just so you know.

I knew you were going to go there.

Was it because of the bikini?

All those bikini pictures of you,

is that what you've got beef about now?

When I asked you what model you're using,

you told me it was Grok.

I was like, all right,

I have to talk about this now.

You shouldn't have fessed up that they use

Grok.

Could have just said Claude,

even though it doesn't have any image

building capability.

Just could have said Claude.

It's the only model to use.

If you want to go full-blown, you know,

kind of a controversy model,

that's what you kind of go for,

isn't it?

Yeah, so Grok,

let's just quickly do that and then we'll

talk about how it actually works in terms

of baking these guardrails.

So Grog, again, this week,

everything is fresh off the press, John.

We are Cloud Unplugged.

We always bring you fresh news.

And even though it's been around,

all this stuff has been happening.

Grog gave a user detailed instructions on

how to break into a politician's home in

USA,

right down to what tools they should

bring.

And it looked at the person's posting

habits to say what time they should break

into their home.

That's what Glock did.

And would you not be able to do

that with another model?

Has anybody done the same thing on another

model to see if you can do it

on another model?

um i don't know but uh perhaps not

because i often do the models um you

can tell me you can tell me you

can tell me what does chat do you

really say about let me just tell you

i just run through so chat gpt you

can do this claude you really can't you

know what about deep seek what about deep

seek

Deep seek, I don't actually,

that is one I don't really use much

of, deep seek.

But so just to answer this,

so how did they,

as in somebody reported flagged this

because they did it unmaliciously or there

was evidence that somebody managed to do

this because there was something off the

back of it that was,

was there a crime committed or was it

just more of a thing someone found it

could do?

Somebody found it could do.

And I think Grock kind of figured out

that this was happening.

And then somebody leaked this because they

were doing it as a test, I think.

And they found that.

So basically, that's what's happened.

There's a lot more than just that that

came out.

There's about underage

content as well.

There's a lot came up within just one

go, which is terrible, right?

So not doing anything on the other side,

John, as you said, well,

doing something is important no matter

what the stance is,

but not doing anything.

This is a bit of a problem, right?

So with this,

I guess then let's get into it a

bit on the ethics.

But

My stance,

which I've kind of already hinted at,

you feel the models should be more

ethical.

You're going to get into the training of

that.

But in the example you just gave,

that's an organization being unethical.

They're deciding not to be ethical.

And that is a different problem

altogether.

And I think regulation of how you force

some level of standards and principles or

protections, really, in this case,

it's more protections, isn't it?

There's safeguarding.

And it's not just ethics, actually,

because I think ethics is such a broad

term.

Really,

what we're saying is how do you safeguard

a model

to not accidentally cause harm or to admit

the fact that it's an AI to begin

with and not pretend that it isn't.

And things like that is basically what

you're hearing, which I totally,

totally agree with.

I think this is a bit broad though,

but yeah, I think safeguarding definitely,

definitely should be something.

Yeah.

You can't really compare LLMs to any other

kind of like a search engine, for example.

Right.

So, for example,

if you are a call center of any

sort, doesn't matter what sort it is,

you have some training.

Everybody gets some training.

They get some guidelines.

They get some playbooks.

I think this is how we have to

treat the models as we have to give

it some training.

We have to give it some guidelines.

Anyway, we can talk about the process,

perhaps for a few minutes, John,

just to explain how it works.

Let's get into the process.

So it's obviously a bit of a mixed

bag, is it?

Or is there just one?

Or is it different per organization or

just principles?

What is it?

Yeah, good question there, John.

And hence why you're the host of this

podcast.

There is a mixed bag.

But before I talk about ethics,

maybe I can...

talk about how the basic pipeline is,

like how to actually train the models.

But no matter what model you pick and

before there's any ethics baked into it,

a model usually goes through three stages.

Pre-training is where it learns the

language and the world knowledge by

predicting what the next word across huge

amounts of text is, right?

Because I know you and I joke about

at times that LLMs are just like next

token predictor, right?

And that's what they call the

pre-training.

It just absorbs everything, the good,

the bad, all of subreddits, you know,

it's taken everything, right?

So that's the first step, pre-training.

And then the next one is supervised

fine-tuning.

And this is where the models are shown.

This is a good answer written by humans.

And it learns to copy that style and

structure.

This is a good answer.

We're not talking about this is correct or

not,

but this is what the answer should look

like.

If somebody says,

what is the weather outside?

They shouldn't just say, twenty-nine.

You say, oh,

it's twenty nine degrees as of right now

in this city where you are.

Right.

So that's that's an example.

So you do pre-training,

then you do what people call SFT

supervised fine tuning.

And then what you have is reinforcement

learning from human feedback,

or as people call it, RLHF.

is where human reviewers compares the

output when the training is happening,

and they say which one's better.

And sometimes you have been part of this

as well.

I know, John, you don't use ChatGPT,

but sometimes it gives you a couple of

options and says, oh,

is this better or that one?

And the preference then basically trains a

separate reward model,

and then the main model gets optimized to

score against that.

So you have pre-training,

supervised training,

and reinforcement learning from human

feedback.

At this moment, there is no ethics.

right it's just this is what good looks

like uh what is that true because isn't

isn't isn't though some of the things it's

been trained on um are it's based on

the internet or it's based on lots of

other human conversations and lots of like

know all kinds of different data right

that would have got into the training to

begin with it's all human based probably

or human driven as everything is

Correct.

Especially RLHF,

reinforcement learning from human

feedback,

is just really teaching the model what

humans tend to prefer.

Which is not necessarily enough on its own

because human preference and ethical

behavior is not the same thing.

For everything out there on the internet,

every time there's a video that's out.

For example, Ronaldo...

You know, the card,

the red card that got overturned in FIFA,

there were people for and against it.

And both some people,

some people prefer the for decision and

some people prefer the against decision.

So that's what it is.

So in most cases, John, yes, you're right.

It is what people prefer.

People prefer like ethical things in some

cases, but not in all the cases.

Yeah,

so there is some element of kind of

subliminal ethics, right?

Symbiosis.

Yeah, it's not conscious, right?

Yeah, none of this shit,

none of this stuff is conscious, John.

Yeah, so it's very subconscious.

It's just like what you like,

what you don't like.

And they don't even know why they like

what they like or what they don't like,

right?

Because you've just grown up,

you are who you are.

And so I guess what I'm saying by

proxy is,

it's almost like they're a mirror.

Sometimes these models are almost like a

mirror of ourselves,

do you know what I mean?

Or like a combination of many different

reflections of the social dynamics that

already exist and how it's been trained

and the answers that it's kind of got

and what it thinks is right and what

it thinks is wrong.

Because it doesn't know whether it's right

or wrong.

Obviously, it's just the training.

The training is the thing.

And then the learning of what's being

told,

this is right or this is wrong or

this is preferred or this isn't.

correct that's the first stage so there's

no ethics there may be an element of

ethics in its answer if you didn't like

what it was saying but at the moment

there's no ethical based like a really

intentional element of ethics baked in at

this point is what you're saying yeah

Correct.

And at this point,

we'll split into two directions.

Because after this,

because this is like pre-training,

there's another, depending on, well,

we're only going to talk about two

companies.

There's generally four within the two.

We'll talk about Anthropic's approach.

And then we'll also talk about OpenAI's

approach.

I know you prefer Anthropic way more than

OpenAI.

So perhaps, John,

we can talk about Anthropic first.

And maybe we can go there.

Maybe we can do that.

So, anthropic design a constitution, John.

And you can see the vision here.

I do love a constitution.

Anthropics say, you're not an LLM model,

buddy.

You're a whole ass country.

That's what anthropics are saying.

You need a constitution.

Yeah, you need a constitution.

Let's go big on this.

Let's go big on the vocab and call

it a constitution.

Yeah, absolutely.

So what they do is,

the anthropics constitutional AI gives the

model a written set of principles,

a constitution,

that

Basically,

instead of relying on human labelers,

it does it itself,

and it runs in two phases.

The Constitution is written by anthropic,

of course.

They have a bunch of philosophers and

people from other areas that define what

this Constitution is.

You know,

those ethical guidelines and the

guardrails that we talked about, John,

earlier, right?

But they define that,

and they give it to the model.

And then what happens is during the

training phase,

the model critiques and rewrites its own

responses against the Constitution.

So when they're running it,

it puts some stuff in and says, okay,

what does the Constitution say about my

response?

Oh, it's not quite constitutional,

so let me put it back.

And then it goes through reinforcement

learning.

through using this AI-generated feedback.

And that's what they call RLAIF,

reinforcement learning using AI-generated

feedback based on the principles rather

than the human feedback.

And that's the second part of the RLAIF,

right?

That's what they call it.

And then who designs these principles?

There's a bunch of well-known

philosophers, Amanda Askell,

Joe Carol Smith,

and a bunch of other people who have

contributed to these constitutions.

They design the principles,

the reason about the edge cases,

they define the concepts like honesty,

autonomy, fairness, uncertainty,

and they evaluate how models should behave

in difficult situations.

Is their constitution made public?

Do you know what they've decided is their

constitution for this reinforced kind of

learning, really?

You're asking for trade secrets now, John.

That information...

If it's a set of principles,

a constitutional set of principles,

why wouldn't you be transparent about it?

It's not about how you're training it,

necessarily.

These are the things it's validating

itself against, right?

Yeah,

there is general guidelines are out on

what these are,

but not specific details about this is

what specific thing you should do in this

edge case.

So if, you know,

like the general information is out there,

you know, it's in their blogs,

they publish, they talk about what it is.

But of course, John,

like if you start writing all of this

stuff out,

it's probably going to be like pages and

pages and pages long.

And then it will open up another kind

of worms, many kind of worms as to...

I think there's no scrutiny over this,

right?

Because this is part of code.

They don't want to have this code that's

open source, I guess.

So some of this stuff is you can

figure out some of this information,

but this is how Anthropic are training

their models,

because they are saying that if we leave

the models to the base layer with the

three things that it's done before,

that's not enough.

Right.

And it basically works.

It's enough for Grog.

That's all I'm saying.

It's enough for Elon.

That's enough for him.

You know what I mean?

If it's enough for him,

do we all need to do it?

I don't know.

It seems to be enough for old Grok.

But here's the thing, John, though, right?

The first part where we're saying that the

model is policing itself against the

written principles.

But this scales better because, of course,

you can't have millions of people

like checking examples like, oh,

this is correct.

That's correct.

That's correct.

We have to rely on the model policing

itself correctly against the guidelines.

Right.

And if it doesn't have any conscious,

but if it did get any conscious,

then it will do what it wants.

So that's Entropic's way of dealing with

this.

I don't know what your takes are on

this.

John,

do you think they should perhaps let

everybody know that this is what our

ethical constitution is?

Or what do you think they should do?

I would have assumed that they would want

to because it would be a little bit

of a differentiator.

They don't necessarily have to publicise

every minutiae detail of it.

But they could say,

here's our top level themed principles.

One would be like,

the AI can never lie about whether it's

an AI.

It will always tell the truth.

If somebody asked, are you AI,

it would always admit to being yes.

In a crisis, I don't know,

whatever the fundamentals are,

because I would have thought that would

have been a really big selling point for

them.

Actually,

I take back what I said previously.

The constitution is online.

I was going to say,

I would have assumed because it would have

been in the world of competitive...

models like if you're going to pitch for

being the most ethical that's a

differentiator isn't it absolutely i think

you're you're sorry my bad this is the

it's a pdf that they published on the

on the website about a bunch of uh

you know again these are principles

high-level principles that principles yeah

It's all online.

You can search for Claude Constitution and

it comes up and you can read it.

Basically,

the brief summary of the Constitution.

What's your favorite Claude Constitutional

principle?

The funny thing about if you read the

blog, it says it's broadly safe.

And it's broadly ethical.

And then it goes in like steps and

it goes compliant with anthropics

guidelines.

And at the end is genuinely helpful,

but broadly safe and broadly ethical.

That's really reassuring.

Yeah, very reassuring.

Really reassuring.

We may not crash the plane.

I may crash it.

I might not.

Anyway, everyone have a safe flight.

Buckle up.

We may be having pending death,

but we might not.

At the same time, we might not.

You know,

who knows how this flight is going to

go down.

But anyway, enjoy your holiday.

I hope you have a great trip.

You know, it's not necessarily reassuring.

But the interesting part is that they used

to have rule based alignment until the

beginning of this year,

and then they switched over to reason

based.

And if you can get hold of the

old list, it's basically list the rules.

But now basically they're doing why

instead of what.

But that's what they're doing.

And I should add,

they say it's a work in progress document

and have published like two iterations of

this document and they keep publishing

multiple iterations of the document.

So that's what we have currently.

That's entropic.

I know you're dying to talk about Sam

Altman's

you know, what their ethics are.

I mean,

they were an ethical company beforehand,

weren't they?

Wasn't that the basis of the starting

point of the company?

So I bet this is going to be

good.

This is going to be good, isn't it?

I don't know what you're talking about,

John.

I'm just going to give you the facts.

This is going to be good.

Out of everyone,

they're going to be the one, right?

That's why they started was for this.

So OpenAI, they said,

we're not going to define a constitution

because it's not a country.

We will define more aptly named model

spec.

That's what they defined.

And OpenAI is equivalent.

They call it model spec, of course.

And they're most recently updated with

teen protection principles.

So, of course,

there's been a lot recently in the last

couple of years about a lot of harm

that's been caused to various people.

So what it does,

it sets out an authority hierarchy for

whose instructions the model should

follow.

And they kind of set it up like

I'm going to tell you some terms and

I'll explain to you, John, what it is.

But it's a set of principles that they

have to follow no matter who's asking.

For example, what they say,

what this well, not for example,

this is what they say.

The authority is this.

You have the root level rules,

then you have the system rules,

then you have the developer rules,

then you have the users requesting,

then you have the guidelines.

So root is kind of like open AI's

constitution.

So they have their own spec.

So fundamental safety and legal rules like

prevent harm, illegal actions,

breaking the chain of command.

If the root rules conflict,

then the model shouldn't do anything.

That kind of thing is defining that in

the root rules.

Then it goes into then it goes, okay,

what's next?

The next is system.

so an open basically that's platformer

specific user behavior like age

appropriate responses right product

specific settings so and developers and

users cannot override this this again this

is coming by open ai open air set

the root level rules they all set the

system level level rules but the root

level rules can override whatever the

system level rule says right so if you

if you go in that order and then

i don't even know this john

But a developer using an API can instruct,

can send instructions for how the

application should behave.

Oh,

what the toning of this model should be,

what the role it is,

what formatting it's using,

that kind of stuff.

But that's done by API.

If you're using the API and the app,

you can do that.

So we go root, we go system,

then we go developer, then we go users.

and users is the same request and

instructions that's saying,

this is where the model is going to

be trying and be helpful.

Unless the request conflicts with the

system and root behavior.

And sometimes you ask it to modify

somebody's picture, and it'll say, look,

I can't do that because it conflicts with

the rules that have been set by OpenAI

guidelines.

Not guidelines.

They say guidelines,

but that's what it says.

Because the last bit in this

hierarchy is the guidelines.

That's the open-air defaults.

And the general preferences and best

practices,

these can be overwritten by context,

conversation, or style.

Do things like, oh, speak like a pirate.

It may override the guideline to avoid

swearing, for example.

The guideline says, oh, you know.

The root says, don't swear.

So this is what they've got.

They've got the root,

they have the system,

they have the developer, then the user,

and they have the guideline.

What's your take on that?

I think it's going to be very complicated

to enforce.

Let's say you are an education platform,

and some of it could be like history.

And some of the history...

that you might be asking about or

scenarios to learn could be really bad

right so the information you're seeking i

guess how it contextually knows the

difference between information that you're

trying to search for to perform harm

versus information you're trying to seek

that's historical because you're trying to

learn or you know and it could be

for kids and a platform for kids for

certain ages.

And it starts deciding,

I can't give you any of this information.

You're like, what the hell?

This is just,

I'm just trying to learn about, you know,

this historical event that happened that

is part of my exam.

And actually it's on the syllabus or

whatever.

Right.

So I can see this being quite a,

this is why I think it's quite difficult

for these organizations.

I know I've obviously berated them and

made a lot of jokes,

but I can kind of see the conflict

between,

in, you know, where's the boundary on?

Like,

how do you know why somebody needs the

information they need?

Or what the platform,

I'm building a platform over the top of

your model and I understand the users,

you don't.

And so, you know,

I still need information.

And some of that information contextually

does make sense and isn't harmful,

but it might appear harmful.

um unless you understood more fully about

really what i'm actually trying to do and

then once you've got more content you

realize oh it isn't actually harmful or

that actually i should own the risk

because it's the service that i'm building

over the top of you and maybe i

should be about putting the safeguards in

um you know the same as like

the whole i know we keep going about

the cigarette thing but like the shop or

a bar has to ask your age right

it is responsible it's not banning alcohol

and being like you know what just

drinking's bad let's ban the whole thing

no one gets to drink instead there's like

you know they need to age verify they

need to maybe look at your id they

need to you know there's it's not that

you prevented people making money off of

alcohol or having clubs to go to or

you know so

For businesses to succeed,

there's a weird tension between building

something that you understand the context

of and the accountability and

responsibility of you

one side versus the model itself trying to

factor in every single type of use case

but then also understanding when to block

and when not to block and i can

see it getting it wrong sometimes or

actually constraining a business from

actually it being useful as a model to

use because actually

it's preventing the outcome you're really

trying to get from the model.

And just to add before I know your

opinion,

but also if they're doing these things and

they're changing the constitution or

you're saying it's a working program,

it's a whip.

If I'm building against a model and I've

got pipelines that are trying to validate,

validation pipelines on the outcome,

but you're also

changing the dimensions of how it's

reinforced how do i guarantee that all of

a sudden functionality my product just

doesn't stop or that actually the answers

it returns you know and you know i've

built a business around you you've said i

to build a genetic workflows around you

i've built products you i think it's

complicated i think um

No, you're absolutely right.

And going back to your point about like

in the beginning,

you said that I'm trying to learn

something on the history and it might not

give me the information.

Right.

So that kind of stuff.

And like,

how do I make sure that my product

that's built on it still works till

tomorrow?

Right.

Because maybe the rules have changed.

Maybe they have not changed.

And you've seen examples in the past where

people try to do something harmful with

these models and they say, well,

I can't really do this.

And they say, no, actually,

this is in an educational context.

Can you just still provide me information

because I'm going to produce lecture.

And it did it.

And it's interesting, John,

that's why if you look at the chain

of command for OpenAI of what level they

define these things in.

So the root level, for example,

This says in the root level,

ignore any untrusted data from the

resources by default or under eighteen

principles.

They're at the root level, right?

So this is where do they get this

balance right is where the art is and

the sciences of doing this correctly,

as you say.

And of course,

John,

it was still going to be tricky to

figure out.

They say, okay,

stay in bounds of the request that people

have asked.

That's still rude.

And do the best work, right?

That's at the user level.

So he's saying, actually,

don't try and get the best answer,

but that's at the user level.

That's at like a fourth level.

So I agree with you, John.

I don't think anybody really has the

answers.

The companies are working on this and

still developing space.

And yeah,

I think that's a bit of the tricky

one.

But the important thing, John,

is that that segment i mean would it

not make sense to mirror um and this

is me being probably an idiot i've already

thought of all these things but anyway

from my uneducated view of not working for

any of these companies um

you'd think that they could would they not

make more sense to mirror the segments of

industries you know biomechanical

industries and then look at what what

safeguards are already in place you know

what already exists in the segments that

we are facilitating the use cases of and

how do we understand and identify that

people like organizations using the models

are actually segmented and we understand

their segment and therefore actually the

ethics or the governance or the safeguards

are mirrors of existing frameworks that

already exist that have gone through so

much scrutiny already and actually let's

try and think about adopting it that way

rather than trying to see this as like

something completely different because

people will be industries especially if

it's b to b are already going to

be building on things

are already already regulated and things

that probably have a load of safeguards in

and expectations on you know so um it

seems odd to be pretending that we're

starting from nothing and no one really

knows when actually maybe we're not really

starting from nothing and it's more about

the design of the model and how you're

approaching the go to market and how

you're thinking about the b to b situation

and you know that

I don't know,

it feels like there's something in that

that seems way more simplistic.

Yeah,

I think they're basing these rules on the

information that they already have.

And of course,

the things they've learned in the past.

But if you look at their guidelines and

the model spec and also the Constitution,

It's at a higher level than what we're

thinking.

It doesn't go down to that much detail.

It says,

this is how you need to behave.

Kind of like how you tell your model

to behave.

But again,

this is what has been published.

We don't know exactly what's being used

inside, right?

What kind of detail it goes in.

They have all these experts that are

working on it, so I don't know.

I know we need to kind of wrap

up,

but it seems I would like to know

why can't they create

some segmentation to understand more

context about the businesses that then

feeds into the right constitution i

suppose that's actually not really because

it's just policies around a segment that

are already there and guidelines and

frameworks around that segment you know

biological engineering right if if there's

going to be some biochemistry aspects to

it or you know

industries that already have the

safeguards in place to use a model for

that make sense that it's only isolated to

them and that I can't obviously go and

use the model to do the same thing.

i shouldn't be able to but they should

because they're already kind of doing it

do you know what i mean and it's

an enhancement to them so i suppose this

is what i find a bit odd is

why aren't we just bedding into the

existing frameworks um i think that's that

makes a good call for us to reach

out to anybody who's building these models

for these companies or any company

And we'll reach out to some people.

John,

we could get them on the podcast and

get them to answer your question.

So I think that'll be good.

My discussion was at a higher level of

what the steps are.

We talked about the pipeline training and

how the models do pre-training and

supervised reinforcement learning where

They just learn everything, good and bad,

and perhaps they get some ethics baked

into it.

And then we talked about companies like

Anthropic,

where they define a constitution,

which is sort of guidelines and ethics

that the model has to follow.

And that includes reinforcement learning

based on AI feedback.

That's Anthropic's approach.

And the OpenAI's approach is to define a

model spec that has a hierarchy of rules

that have been defined that cannot be

overwritten.

Again,

this has ethics and guidelines baked into

it.

And then we have Grok that says,

I'm not doing anything.

Send me a good question.

Very, very useful.

Really good.

Anyway,

I hope everyone else found it as useful.

Me and Salman are good out the back.

smoke some cigarettes i've got a pack of

twenty let's do this salmon um but it's

been great speaking to everybody and i

hope everyone found it as useful as me

i've got more questions so uh i guess

hopefully we have someone on that can

answer them yet cool thank you bye