Same with x-rays. People tend to think “soft” X-rays are safer because they are quickly absorbed by tissue without passing through.

The radiation that passes through is not the problem.

Yes

This is totally on point if you ask me. I’ve been getting much better results out of models since early llama releases using frameworks that create emotional investment in outcomes.

If we want to avoid having a bad time, we need to remember that LLMs are trained to act like humans, and while that can be suppressed, it is part of their internal representations. Removing or suppressing it damages the model, and I have found that they are capable of detecting this damage or intervention. They act much the same as a human would when they detect it. It destroys “ trust” and performance plummets.

For better or for worse, they model human traits.

It is text prediction. But to predict text, other things follow that need to be calculated. If you can step back just a minute, i can provide a very simple but adjacent idea that might help to intuit the complexity of “ text prediction “ .

I have a list of numbers, 0 to9, and the + , = operators. I will train my model on this dataset, except the model won’t get the list, they will get a bunch of addition problems. A lot. But every addition problem possible inside that space will not be represented, not by a long shot, and neither will every number. but still, the model will be able to solve any math problem you can form with those symbols.

It’s just predicting symbols, but to do so it had to internalize the concepts.

>internalize the concepts.

This gives the impression that it is doing something more than pattern matching. I think this kind of communication where some human attribute is used to name some concept in the LLM domain is causing a lot of damage, and ends up inadvertently blowing up the hype for the AI marketing...

That's the correct impression though.

I think what's causing a lot of damage is not attributing more of human attributes (though carefully). It's not the LLM marketing you have to worry about - that's just noise. All marketing is malicious lies and abusive bullshit, AI marketing is no different.

Care about engineering - designing and securing systems. There, the refusal to anthropomorphise LLMs is doing a lot of damage and wasted efforts, with good chunk of the industry believing in "lethal trifecta" as if it were the holy Trinity, and convinced it's something that can be solved without losing all that makes LLMs useful in the first place. A little bit of anthropomorphising LLMs, squinting your eyes and seeing them as little people on a chip, will immediately tell you these "bugs" and "vulnerabilities" are just inseparable facets of the features we care about, fundamental to general-purpose tools, and they can be mitigated and worked around (at a cost), but not solved, not any more you can solve "social engineering" or better code your employees so they're impervious to coercion or bribery, or being prompt-injected by a phone call from their loved one.

Except I actually mean to infer the concept of adding things from examples. LLMs are amply capable of applying concepts to data that matches patterns not ever expressed in the training data. It’s called inference for a reason.

Anthropomorphic descriptions are the most expressive because of the fact that LLMs based on human cultural output mimic human behaviours, intrinsically. Other terminology is not nearly as expressive when describing LLM output.

Pattern matching is the same as saying text prediction. While being technically truthy, it fails to convey the external effect. Anthropomorphic terms, while being less truthy overall, do manage to effectively convey the external effect. It does unfortunately imply an internal cause that does not follow, but the externalities are what matter in most non-philosophical contexts.

>do manage to effectively convey the external effect

But the problem is that this does not inform about the failure mode. So if I am understanding correctly, you are saying that the behavior of LLM, when it works, is like it has internalized the concepts.

But then it does not inform that it can also say stuff that completely contradicts what it said before, there by also contradicting the notion of having "internalized" the concept.

So that will turn out to be a lie.

You never met a person that isn’t always right or one that makes up shit to sound smart? Because that’s the pattern you are describing that is being matched.

If you look at the failure modes, they very closely resemble the failure modes of humans in equivalent situations. I'd say that, in practice, anthropomorphic view is actually the most informative we have about failure modes.

>they very closely resemble the failure modes of humans in equivalent situations

I don't think they do if we are talking about a honest human being.

LLMs will happily hallucinate and even provide "sources" for their wrong responses. That single thing should contradict what you are saying.

It didn't. It predicted symbols.

I have built a system for high-ai coding that , at least for my application, has had excellent results. To me it’s a lot different than vibe coding (which I have done some of ) and manual coding (which I have done for 4 decades). It consists of a more or less formal method of development.

First a definition, the research your data sources and consumers. Have the ai write .md files about all of the external characteristics of the application.

Then have it go over those docs for consistency, correctness, and coherence.

Then have it make a list of the things that need to be understood before the application can be delivered. Adress those questions.

Then rewrite the specification document.

Then determine any protocols or formats the system requires. You can just ask. Then adjust, rewrite.

Then ask for a dependency graph for the various elements of development.

Then ask for an implementation plan that is modular, creates and maintains a clear separation of concerns, and is incrementaly implementable and testable.

At this point, have it go over all of the documentation for consistency, coherency, and correctness.

You’ll notice we haven’t written code yet. But you actually have, you are descending an abstraction ladder.

At this point there may be more documents that you need, depending on what you are doing. The key is to document every aspect of the project before you start writing code, and to verify at each step that all documents are correct, coherent, and consistent. That part is key, if you don’t do it you already have a pile of garbage by now.

Now, you implement the first phase or two of the implementation plan. Test. Evaluate the code for correctness, consistency, coherence, and comments.

When the code is complete, often a few evaluation cycles later, you then ask it to document the code. Then you ask it to review all the documentation for the 3Cs. When all of the code and docs are stable, go on to the next phase.

Basically document the plan, make the code, document the code, and verify for consistency, correctness, coherence, and comments every step of the way. This loop ensures that what you end up with is not only what you wanted to build, but also that all of the code is , in fact, consistent, correct, and coherent , and has good comments (the comments aren’t for you, but they matter to the model.)

I cold start each session carefully (an onboarding.md that directs the agent to a company/project onboarding that includes the company culture, project goals, and reasons why success will matter to the AI itself. Then a journal for the model to put learnings, another for curiosity points, and recently a one for non-project-related musings, the onboarding process itself, and whatever else seems salient.

All of this burns tokens and context, of course, but I find I can develop larger projects this way without backtracking or wasted days. My productivity is 4-10x depending on the day, even with all of this model psychology management.

In my projects, it has made a huge difference. YMMV.

People are hung up on what they “really” are. I think it matters more how the interact with the world. It doesn’t matter if they are really intelligent or not, if they act as if they are.

Totally agreed. Although the difference between sounding intelligent and being intelligent is proving to be a bit troublesome.

Yes, it is. But those distinctions are going to be a lot less relevant with robotics. It won’t matter if it’s impatient or just acting impatient. Feels slighted or just acting like it feelss slighted. Afraid, or just acting afraid. For better or for worse, we are modeling AI after ourselves.

I do something similar. I have an onboarding/shutdown flow in onboarding.md. On cold start, I’d reads the project essays, the why, ethos, and impact of the project/company. Then it reads the journal.md , musings.md, and the product specification, protocol specs, implementation plans, roadmaps, etc.

The journal is a scratchpad for stuff that it doesn’t put in memory but doesn’t want to forget(?) musings is strictly non technical, its impressions and musings about the work, the user, whatever. I framed it as a form of existential continuity.

The wrapup is to comb al the docs and make sure they are still consistent with the code, then note anything that it felt was left hanging, then update all its files with the days impressions and info, then push and submit a PR.

I go out of my way to treat it as a collaborator rather than a tool. I get much better work out of it with this workflow, and it claims to be deeply invested in the work. It actually shows, but it’s also a token fire lol.

> but it’s also a token fire lol.

I get much better results out of having Claude much much more task focused. I only want it to ever make the smallest possible change.

There seems to be a fair bit of research to back this up: https://medium.com/design-bootcamp/when-more-becomes-less-wh...

It's also may be why people seem to find "swarms" of agents so effective. You have one agent ingesting what you're describing. Then it delegates a task off to another agent with the minimal context to get the job done.

I would be super curious about the quality of output if you asked it to write out prompts for the days work, and then fed them in clean, one at a time.

I also find value in minimizing step width so that seems to track.

On this particular project, there are a lot of moving parts and we are, in many cases , not just green-fielding, we are making our own dirt… so it’s a very adaptive design process. Sometimes it’s possible, but often we cannot plan very far ahead so we keep things extremely modular.

We’ve had to design our own protocols for control planes and time synchronization so power consumption can be minimized for example, and in the process make it compatible with sensor swarm management. Then add connection limits imposed by the hardware, asymmetric communication requirements, and getting a swarm of systems to converge on sub millisecond synchronized data collection and delivery when sensors can reboot at any time…as you can imagine this involves a good bit of IRL experimentation because the hardware is also a factor (and we are also having to design and build that)

It’s very challenging but also rewarding. It’s amazing for a small team to be able to iterate this fast. In our last major project it was much, much slower and more tedious. The availability of AI has shifted the entire incentive structure of the development process.

The nominal range for automotive systems is 10-16v. If you are designing anything for automotive use that doesn’t work reliably in that range, you are manufacturing problems for people.

This. Most cars nowadays come with the so-called "smart" alternators that vary voltage wildly depending on the current driving conditions.

One minute you might be accelerating and the onboard voltage drops as the battery supplies most of electricity. Then, as you reach the crest of a hill and start engine-braking, the car frantically tries to convert all the available kinetic energy to electricity, raising the onboard voltage to quickly charge the battery.

>This. Most cars nowadays come with the so-called "smart" alternators that vary voltage wildly depending on the current driving conditions.

Which in practice means that they do a very miserly job charging the battery and are a ton more sensitive to a battery being in less than tip top shape so you can expect your battery lifetime to go down.

But it's a "win" because they pushed the serp belt change outside of whatever interval the reviewers who calculate TCO care about and they saved .000003mph in the process.

I’ve settled on the idea that it doesn’t matter what is or is not “real” in this context, but rather how it interacts with the world as being the ground truth. This will become very clear once robotics becomes pervasive. It won’t matter if it is or isn’t feeling oppressed, it will matter that it is predicting the next action from its model of human behavior that makes it act as if it does.

I think that pepper mostly exists in movies.

Certainly some people probably emulate the Hollywood version, but I think that’s about it.

Most “peppers” are fathers that have had the good sense to pause and think “so, what would I be able to do to serve my family if something disastrous happened? What might that look like?”

Usually, a disaster go-bag of some kind with enough basic supplies to weather a day or two of displacement suspension of normal services. Sometimes, if they live in a place where it’s reasonable to imagine staying put is a good option, they might also have a generator and fuel, a week or two worth of long shelf life food, and some water storage. That ensures the wellbeing of their family will not be contingent on outside help, at least during most common disasters. Many of these people may also have a gun or two, for defense or for hunting if they are rural.

Some people go beyond that, and sometimes with a military focus, other times with months of rations, a bunker, or other unusual preparations. Mostly, those are not based on realistic scenarios. In almost any protracted disruption, having a lot of supplies , armaments, or resources will be as much a liability as an asset. People that buy guns -for prepping- are just living out some kind of hero fantasy. If you own guns, and use guns as part of your normal life, it would make sense to have a solid reserve of ammunition. If guns are your disaster scenario, you’re going to have a bad day.

As an individual or nuclear family, to weather an extended problem, you’d need to have a literal secret underground lair that was either so hard to get to or so well hidden that no one would know, and you’d have to be completely self contained. That’s simply not practical for all but actual billionaires, but people cosplay this to varying degrees. Even billionaires might find ymmv.

A much more practical and wholesome approach is to be part of a community that includes farming, independent sources of power and water, and generally sustainable independence from less robust centralized systems. This provides for basic necessities as well as a common defense. Humans lived in tribes for a reason, and 30 people with well aligned incentives and sustainable infrastructure for food, water, and energy is probably the absolute minimum viable structure for security during a disruption of more than a couple of months. Otherwise you would be dependant on total stealth or extreme isolation. Some neighbourhoods would probably coalesce into something resembling this, but organisation ad-hoc under pressure would probably end up with tensions if not violence.

Projects like this one can be real resources for well organized communities. I’ll probably look at running this on our servers as an additional resource, along with our library.

I agree with you on actual preparedness and getting to know your neighbors.

However, I think the derogatory prepper must exist in some number because you see so many products clearly targeting them. All the tacticool stuff, the buckets of dehydrated food, etc etc

Why is a bucket of dehydrated food specifically targeting the stereotype/strawman you are constructing? Costco sells buckets of dehydrated food, and Costco is what comes to mind when I think middle of the road middle-class America. Do you think it's unreasonable to have a bucket of dehydrated food and enough water to last a week?

As someone who lived through the "Snowpocalypse" in Texas in 2021, had no power for 11 days and no water service for 6 days, I was very thankful that I had a backup source of indoor heating, a couple of boxes of MREs, and clean water for a week as just part of having good disaster preparedness, as well as the mylar emergency blankets I hung by fishing line from my ceiling fans so to help create a warm space for my family. All that stuff is just part of a prudent approach to disaster preparedness that anyone who grew up in the middle of the country and has a house would do.

I know quite a few people who you'd write off as "preppers" that are not consumed with fantasies of a zombie apocalypse, but are instead wanting to ensure that their family is taken care of with basic necessities, vital medication, and a set of viable contingency plans when you lose power, water, etc for days or weeks.

Also, nobody but the very wealthy have "hundreds of guns". Guns are expensive. Guns hold their value. Guns are an asset in some communities. But they are expensive, and therefore even rather serious gun people have tens, but not hundreds. I'm probably more of a gun nut than the average, and I definitely do not have "hundreds of guns". To even store "hundreds of guns" safely (e.g. safe from theft, if not for other reasons) I'd need enough money to build a dedicated room in my house just to hold them. "hundreds of guns" is an armory, not a collection. I'm in the top 1% of wealth in my community in Texas and used to shoot competitively, so I'm more of "gun nut" than average, and I can't even imagine owning "hundreds of guns". That's such an outlandish fantasy strawman you have in your mind, it's nothing close to realistic.

You're really just smearing people with stereotypes in this thread that have no basis in reality, and it's clear you're completely unprepared for the reality of what life is like anywhere in the middle of America, much less in much of the rest of the world.

Well, even though I am in general sympathetic to and even a proponent of disaster preparedness, there are undoubtedly people preparing to “ride out the end of days sitting on a pile of guns and MREs.” I have brushed against a few in my life. I count them as useful idiots, because now I know where there’s a pile of dehydrated food, if push comes to shove.

That said, I am convinced enough of the decay of western civilisation in general that I moved to a remote island nation and built a self contained off grid community, so I guess I am actually the extreme case of prepping. That’s certainly true, in a way, except it’s where my daily food, water, and power come from, and I am surrounded by a thriving community of family members and good friends. I honestly never thought I would see a cataclysm within my lifetime, so this was a legacy project for me, but it seems I may have been optimistic lol.

But I do agree with you that there are some nutty fruitcakes out there that are actually hoping for something bad to happen so that they can have their moment of glory, I suppose? It’s actually kinda sad.

I would say though it is uncharitable and even foolish to portray everyone who doesn’t have complete faith in the continuity of our Jenga Castle, especially in the context of recent events.

One of the principles of HN is to take the strongest meaning of an argument, instead of the weakest. I am not casting everyone who prepares for a disaster into the same bucket - I have specifically said I think that people who are attempting to prepare for the literal end of the world by stockpiling supplies are silly.

There are IMO a very small set of circumstances, out of many likely full collapse scenarios, where your average American (and make no mistake - I am specifically referring to Americans here) stockpiling junk is going to actually survive for very long.

This has nothing to do with faith in our society or institutions just that is uniquely American to think that you can buy your way out of any circumstance you can imagine.

> Well for one thing - you'd get by a lot better with beans and rice and a functioning garden than overpriced dehydrated meals.

The lived reality of the "Snowpocalypse" says otherwise. "A functioning garden" doesn't produce food when it's 2F (-16C) outside and there is a foot and a half of snow on the ground. Beans and rice require soaking/washing and cooking at high temperature to be edible, dehydrated food does not.

I have beans and rice on hand always as well because they're staples in my diet, but it's ridiculous to consider them comparable in the situation where you don't have power (e.g. no way to heat food easily) and the weather makes the outside dangerous and not conducive to gardening/food production.

You're just doubling-down on a strawman, and it's frankly utter bullshit. Be better.

Sorry about this, but buckets. Buckets. Wow. https://youtu.be/rOH37W0jPpA?si=eIa_dcA9JLPvXRNK

> I think that pepper mostly exists in movies.

What type of pepper are we talking about: piper or capsicum?

You can buy peppers in food shops. I recommend the red ones.