Curiosity, Smartest Person in the Org, and Claude vs. GPT

Welcome to wake up with AI, the podcast where human powered meets AI assisted. Join your hosts, Chris Carillon, Niko Lofakas, and George b Thomas as we dive deep into the world of artificial intelligence. From the latest AI news to cutting edge tools and skill sets, we are here to help business owners, marketers, and everyday individuals unlock their full potential with the power of AI. Let's get started.

Good morning. LinkedIn, YouTube, Spotify, Apple Podcasts, wherever you might be listening, or watching. Happy Friday, December 6th. It is time to wake up with AI here with George and Nico celebrating day 2 of Shipmas, from Chat GPT and friends. How you fells doing today?

Doing great. Shipmas day 1, exactly what I expected. So in terms of, what what is going to be releasing over the next 12, already 1 for 3. Got Orion yesterday, o one. So it's the, the full version of that, and, very excited about it.

Yeah. I'm doing good. How are you doing, Chris?

I'm doing good. I'm doing good.

It's been

an interesting week for sure, and we're just getting started with the month. Man, breakneck next speed here. That's why we're showing up every day. What was day 1, Niko? Tell us about what is this like?

There's a pro subscription now charging all kinds of money.

There are a couple aspects to it. So there's o one is the official release. And what is this? It's basically if you've been using o one preview, o one is just I mean, it's crazy better. It reasons much faster and the, like so the speed of the reasoning is greater.

The capability is greater, in my opinion. And I think that overall, it's definitely I I don't know how they compare it. They do say that the cost of it on the API end is higher than gbt 4 o. So that's a bit interesting in that respect, but you know, given the higher reasoning capacity, it kinda doesn't surprise me and I think that's gonna scale down over time. So people who have a Plus subscription now, you have access to 01 and 01 Mini.

They are soon to release, if they haven't already, a pro level subscription. Difference there is it's slightly better, the o one pro model. But when I say slightly, I I just mean that when you're looking at the scales. Underneath the hood from Altman

This and for scientists, engineers, coders, we think they will really love this model. I'd like to show you quickly about how it performs. So you can see, the jump from GPT 4 0 to 0 1 preview across math, competition coding, GPQA, Diamond. And you can see that o one is a pretty big step forward. It's also much better in a lot of other ways, but raw intelligence is something that we care about.

So what we're looking at is a chart that is showing competition level math, competition coding, and PhD level science questions. And on the, vertical is accuracy for competition math, for competition code is, the percentile of it. And, well, I should say for all 3 actually is is what it is. It's the percentile of accuracy for for the model and then the x is just which model it is. So for competition math, g b t 4 ranked in at 13.4% accuracy.

01 preview came in at 56.7 percent and 01 is 83.8%. Competition code, gbt40 came in at 11%, 01 preview was 62%, and 01 is 89%. This is the most important one. For PhD level science questions, GPT 4 0 came in at 56%, o one preview was 78.3%, o one is 78%, and expert human is 69.7%.

In particular, is an area where people are using the model a lot. So, in just a minute, these guys will demo some things about A1. They'll show you how it does at speed, how it does at really hard problems, how it does with multi modality. But first, I want to talk just for a minute about the second thing we're launching today. A lot of people, power users of Chat gpt at this point, they really use it a lot, and they want more compute than $20 a month to buy.

So we're launching a new tier, Chat gpt Pro. And Pro has unlimited access to our models, and also things like advanced voice mode. It also has a, a new thing called o one pro mode. So o one is the smartest model in the world now, except for o one being used in pro mode. And so the hardest problems that people have, o one pro mode lets you do even a little bit better.

So you can see at competition math.

So competition math, o one is at 78.3%, and PhD level science is 75.6%. O one pro mode on competition math is 85%, 85.8, so almost a full 10% higher. And then on PhD level science questions is almost 80%, 79.3, so another 5% higher.

It may look small, but in in complex workflows where you're really pushing the limits of these models, it's pretty significant. I'll show you one more thing about pro about the pro mode.

Same spectrum that we were looking at before when it comes to accuracy, reliable accuracy. This is showing the worst 4, basically. So it's like the worst it could possibly do. At competition math, the accuracy level of 1 is 66.7. The worst 1 pro could do is 80%.

Competition code, 64.2 for 1. 01pro is pretty much 75%. PHD level science questions: o one's worst is 67.2, which if you remember is nearly the same as the best human. 01pro, at its worst, is 74.2%, which means at its worst, it is better than the best human. That's the difference in short.

So you get that level of performance, if you're willing to pay, I think it's like $200 a month or like $300 a month, something like that.

It's 200.

200 a month. Yeah.

I was literally trying to figure out how to upgrade last night.

Yeah. I don't

I couldn't figure it out.

Like I

don't know. I already have a team's plan, and I was like I went to my billing, and there was, like, there was no give me more button, Sam. Like, Sam, if you're if you're listening to this, if you're watching, like, a give me more button in my billing area would have been great because you woulda had another now I don't know if it's launched to everybody, but I literally started watching YouTube videos, and this other dude on YouTube was able to, like, get to a screen where he could upgrade. Now maybe it was he just started a new account. Because I even thought about, like, maybe I just go start a new account.

And I'm like, no. I like, I want my I want it to be part of my team's plan and because I was like, listen. I'm invested, and I've got some, like

I hate I'm so sorry, man.

I'm like Pretty large things on there.

Because, like, I I don't wanna be one of those guys, but, like, I'm one of those guys too. I think it's an early adopter thing, to answer your question.

Yeah. Yeah.

Yeah. If you were, like, serious early adopter, you'd definitely have access to it now.

I mean, I'm definitely not an early adopter, and I've used, GPT way less. I I have to imagine, George, it it's related to the team account.

I think so. Yeah. I think it's related to that I'm on a team plan versus, like, an independent individual, which is why I said I I thought about going and just doing an individual plan, but I'm like, let me just play with o one for a little bit anyway and and curb my wanting to spend money because I got to this screen as an individual human, but I'm like, I can't tie it to my already existing plan. Yeah.

Yeah. So for business, it's it's not an option.

Which is weird.

I'm sure it will be soon enough, but it's another I

wonder I wonder what that enterprise tag is gonna be. I mean, if it's gotta be 10 x. Right?

Yeah. I don't know what the enterprise I don't need enterprise, but I'm willing with what we do to see the investment, you know, but but we'll see. We'll see.

Got a contact for quote there.

Yeah. It feels like very old school. Like, here's this modern technology and an historical sales process.

I found a, well, I found it's you know, comes all through my feed, but a video of, like, sort of collaborative video between Morgan Stanley and OpenAI. And the clip that stuck out the most to be that I think sort of speaks to not only, like, what we have now, right, but, like, what's coming, what's, you know, what people, I guess, should be thinking about when it comes to where this is this is all going and, like, how this is all affecting business.

My name is Jeff McMillan. I am the head of firm wide AI here at Morgan Stanley. We in financial services are an incredibly regulated, business. And while many people say to us, how did you get out in front of this? And I would respond by saying large financial institutions like Morgan Stanley are not new to complex regulatory environments.

We had to work in deep collaboration. I give open act tremendous amount of credit. That experience, I think, has been really helpful and has allowed us to actually accelerate our efforts. The question was, where do you start? And we wound up in a spot where we agreed that we would focus on the intellectual capital that sits between the firm and our financial advisors.

What this technology does is it makes you as smart as the smartest person in the organization.

That was the quip I was looking for, but, you know, coupled with the second half of that is sort of a perfect way of illustrating Jamie Dimon's point that he made a couple months ago when he said, I don't care if you know anything about banking as long as you know something about generative AI. Like, if you know how to use chat g p t, then please apply for a job. That's sort of where this level of adoption is going. I think the most profound statement really outside of, like, Shipman's was amazing, but honestly, it came out of that interview was that Sam said, you know, middle of next year, we are going to start seeing things that it'll be such a a shockwave that people who aren't paying attention are going to be saying to themselves, wow. I thought that was just, like, completely impossible and would never happen.

And, like, to see the test results per se, even Samsung's own unabashed statement that o one is the smartest model in the world now. And it probably is. Again, like, if we're talking about, like, o one versus quad in terms of model strength, smartness, all that kind of thing, we still haven't seen Opus 3.5. I say it's it's definitely the most powerful for now.

Here's the thing where my brain goes with this, and it's an interesting, like, true life fact that you'll find as well. I've met the smartest people, and they are just not good at communication. And I've met some people that are kinda smart, man. They're great communicator. And as I was watching the same video that you put up earlier where it's the Sam and, you know, the 3 other folks there, I had a couple of thoughts.

1, okay. It's super intelligent, but it's still not writing like a human. And I watched a couple other YouTube videos where people were using o one and and looking at it versus Claude and how Claude is when we're talking about generative marketing, like copy, this supposedly smart nerd is not able to communicate as humanly as I would like it to communicate. So there's there's that. And I am not trying to be funny, and I am not trying to be rude when I say this next thing that hit my brain when I watch that video of the launch of the 12 days of Chipmas.

I had to rewind it a couple of times and ask myself, is this generative video? Is this not even them on the screen? Because if you pay attention to, like, hey, Jen and hand gestures, and if you also watch the, like, the 3rd guy from the left when he communicates, it's like, how about number like, it's super, like, just like and maybe that's the way he communicates. I literally was like, wow. Are we being punked right now?

Like, is the 1st day of shipmas actually Sora? And, like, these people aren't even at that dang desk. Now I don't know if something will come out at the end of this of, like, by the way, but I'm like, wait. I don't know.

Like, that

background and the Brilliant. Anyway If they get to

the end of 12 days and they posted a video every day and then the launch of Sora and, like, by the way, all 12 videos

I mean, I don't I don't know. But but you know what I'm saying? Could.

Right? If they were comfortable, that would be a brilliant move.

Teach it to be fast. Teach it to be smart, but teach it to also, like, create better, write better, be more human. Like,

To me, I don't know. Like, this this is just the way that that I see it because it's I don't know. That's the way I see that. It's kinda funny. I was talking about that with Claude.

He was asking me this morning, like

You guys talk to Claude way too much. I know I do too, so never mind.

Yeah. And that's what I tell people who are using the tool too. Every conversation is, like, it's brand new. Right? You're talking to a brand new person, so to speak.

But, again, there's no context. There's just it's just brand new. They have no clue what it is that you wanna do. They have no clue how it is you want them to respond, you know. And a lot of that also, George, is based on your custom instructions.

Oh, I know. Trust me. My custom instructions are dope. But

So, like, yeah. I I can understand that by default, it doesn't sound like marketing content or by default can't, like, write it that well or or what have you. But by default, a human that graduates college with, like, a general a general degree doesn't really know how to write marketing content that well. And they can have, like, of course, they have, like, better conversational English. Right?

But that's also because they've had more conversations with other humans.

It speaks to the nuance of that last comment in the second video where everybody in the organization is as smart as the smartest person. And I was already thinking about how all the metrics that you were showing are still based on the human input that is happening. And I'm sure they have controlled scenarios that actually, like it it can only get that level of consistency if the AI starts handling, you know, the inputs. But, again, come back to the human inputs. But then, like, that statement as smart as the smartest person, we already know there's book smarts and street smarts.

And the street smarts relate very much to the communication that George is talking about. And if we know that organizations and humans as a whole suck at communication to the tune of, like, less than 5% of organizations have communication training programs, and it's not something we're taught in schools. And if and if we're the ones that have created and trained AI, is it gonna get better at that until it reaches the place where it's the one teaching itself? Because we ourselves have not gotten very good at teaching communication at scale. Obviously, there's pockets of people, but if as long as the training model is trying to take into account everything

It's interesting that you say that because I do feel like I spend a lot of my time and a lot of my, like, custom instructions and things is I'm teaching it communication skills. And when I give prompts that have communication skills specificity, it does a lot better job. So it's interesting that you bring that up. Now I wanna bring something up because you brought it up, Chris. And in the video, it says the smartest person in the room.

Planet could be a room. The planet could be a room. And and what I mean by that is you can pick the people that you wanna be as smart as. And so I'm just gonna, share my screen real quick here for a second because I want everybody to see a chat that I started yesterday because I wanted to be smart about something that I knew that I believed in and that I thought was a good idea. And so I literally started a chat like, hey.

I want you to research the story brand. This is the exact story brand that I'm talking about, and I want you to pay attention to all the best practices and get it in here. Then I loaded it up with some information like, hey. Look at all this information for these other things that I want you to pay attention to because I'm working on this thing that is superhuman framework. And so I'm, like, feeding it context, and I'm I'm teaching it now.

Let's let's completely rewrite the copy that I have in this one place now that you know this other thing and that you're paying attention to this other information that we have. And so I just go through this process, and, like, I'm just giving it a metric butt ton of information and context and the way that I wanted to communicate. And so then, boom, out the other side, we pop out what is actually gonna happen. So we've got the hero section, the problem. We've got the solution.

We've got the process. We've got all these sections of information that is, by the way, Aniko, Zope marketing, like, centered copy with a strategy that I basically said, hey. I want you to look at everything that you know about this and let me use it. I didn't go watch 12 YouTube videos. I didn't go spend $10,000 on a program.

I didn't, like, go hire somebody to write the copy. I said, this is what I want you to do and how I want you to do it because smart people put it out in the world, and I wanna leverage it for something that we're building. You can be as smart as the smartest person if you're smart enough to use the tool that helps you be smart as the smartest

person. Like, that's kind of it too is to use so because that statement makes it seem as though there's some, like, level of difficulty with, like, using any of these models. And then There is.

There has

to be a great communicator. Old. You have to be able to put the pieces of the puzzle together. Anyway, I'll shut up.

So there's 2 things. 1, you perfectly illustrated why there is an actual difference and how there is an actual difference between these models. Because what you were trying to do probably would have been 10 x easier using Claude than it would have been using GPT. Right? Because Claude already has this sort of natural understanding of writing, natural understanding of speaking, etcetera and so forth.

Now to the second part, yeah. You do on one hand, you could, like, be the person that knows. On the other, you could just be inquisitive, and you could just ask your way to the answer. And I'm not even joking when I say it. I've done that multiple times.

Even when it comes to working with software or whatever it might be, I'll just ask and say, like, yeah. What do I need to do in order to do this? Or how would I best go about doing this? Right?

Curiosity is a superpower. By the way, curiosity has always been a superpower in a world of AI. I mean, I don't know if there's a such thing as a super superpower, like an ultimate power. I'll use the young lad's, terminology. AI can make you OP.

Is that a term that's back?

I don't know. I don't know.

Jordan, the

old part of the I'm I'm the I'm the old guard.

Overpowered. Overpowered, I think, is is what that is. I heard Mike

I heard Mike forget to yell that for you. No.

That definitely is overpowered.

Yeah. Okay.

Not down with OPP. Just OP.

Oh, man. It's interesting, Nico, because, again, how many people are actually good at asking questions like that where there'll still be value? Like, if George would have approached that, like, I wanna write not to, belittle it too much, but I wanna do story mode on this stuff. How like, there's so many other things. Like, we should own the marketing space.

There's so many frameworks, so many things that have gotten attention and, you know, that that could have shown up. And now you're doing the work of, like, that doesn't lie with me. And, like, I love being able to either correct the course or or manage the course from the get go if I already have a belief or a a an understanding of something that I know the outcome will be better if we go that route. But I think an often example when I'm trying to get to a new understanding or or learning is when I'm trying to get to a new understanding or or learning is having to ask Claude to say, no. I don't want the general human knowledge base to be your answers here.

I want the AI native. That phrase is one of the phrases I like to use to get like, give me the AI take here. Right? And that changes the conversations, like, drastically every time.

So curiosity and the power of questions and what you guys are talking about. I just wanna pull this one little piece up here because it's a one of 37 pages here. Unique value, proposition and problems and the personas and well, it's just it's just a document. Okay. It's it's a big doc.

But here's the thing, the power of curiosity. Okay. We've been working on this document along the way with this conversation. Please look at it and tell me what areas we should be thinking about to create the superhuman framework as an amazing resource for these humans as well as make it a thriving business model in the future. What might be the gaps I'm missing?

My blind spots, if you will. I don't know I know everything. I don't think I know everything. I know I don't know a lot of things. Curiosity and understanding that you might be dumb combined together.

Like, how smart am I? How dumb am I? I don't know. Let's ask the question. And then that's when you get to the magic place of like, oh, yeah.

That'd be a great idea. That's the other thing. I hope people are leveraging it for, like, just these moments of like here's the thing. In organizations, you're afraid to ask the question because you think you're gonna be judged. Ask the damn question.

In AI, ask the question because you'll be like, dang. Let's do that and that and maybe a little of that over there too.

And while we're at it, let's wake up with AI. Have a great weekend, everybody.

That's a wrap for this episode of wake up with AI. We hope that you feel a little more inspired, a little more informed, and a whole lot more excited about how AI can augment your life and business. Always remember that this journey is just the beginning and that we are right here with you every step of the way. If you love today's episode, don't forget to subscribe, share, and leave a review. You can also connect with us on social media to stay updated with all things AI.

Until next time. Stay curious, stay empowered, and wake up with AI.