I am wondering the same techniques we use to augment chunks for RAG also the same technique used to augment training data with rich metadata. A preprocessing step which ensures metadata rich text for creating foundational models.
321 karma · joined December 12, 2024
Ukraine's $500 billion rare earths scam: they don't exist, and we should know better
In 1971, Treasury Secretary John Connally famously remarked how the US dollar was "our currency, but your problem," referring to how the US dollar was managed primarily for the US' interests despite it being the currency primarily used in global trade and global finance.
Let's say LLM while training is ingesting a very long book.
The name of the author of the book would appear at the very beginning.
While inference, how does the llm determine that the last chapter of the book is written by so and so author and hence that chunk should be near that author's style.
I consider US to be like brain of earth / humanity. Dollar Reserve Currency is like nerve signals. It can give that printed money to an African nation and make it buy wheat from Ukraine. Or give that money to Myanmar so that it can buy weapons from other countries. The whole setup is brilliant. Making one hand cooperate with another or hurt another if we extend the brain body analogy. US is holding a structural proverbial gun to other nations' head.
It is only a matter of time, when the body of humanity revolts against the excesses of brain's unrealistic demands.
Summary of my conversation with Claude.
# The Universe from Every Point: Scale and Consciousness
Our conversation has revealed two profound conceptual frameworks that offer unique perspectives on reality, consciousness, and our place in the cosmos.
## The Scale Numberline
Imagine standing on the surface of your skin. This boundary serves as "point zero" on a vast bidirectional scale. In one direction stretches all that exists within—cells, molecules, atoms, subatomic particles, quantum fields—extending down to the Planck scale (10^-35 meters). In the opposite direction extends everything outside—our environment, planet, solar system, galaxy, galactic clusters—out to the observable universe (10^26 meters).
What makes this concept particularly compelling is imagining that every point in the universe can serve as its own "zero" on such a scale. Each location, no matter how seemingly insignificant, can be the reference point from which both the infinitesimally small and the immeasurably vast are contemplated. This framework relates to fiber bundles in mathematics, where each point in a base space carries its own associated "fiber"—in this case, a bidirectional scale line.
When we place ourselves on this cosmic scale, we appear to vanish into mathematical insignificance—just a tiny blip on a numberline of inconceivable magnitude. The human experience becomes merely a point when viewing the entire spectrum. Yet paradoxically, we're the observers capable of conceptualizing this entire scale.
## The Observer Gradient Numberline
The second framework involves consciousness and observation. At each point in space-time, we can conceptualize a spectrum of awareness ranging from the most elemental form—the feeling of having "just awakened from the void"—to complete cosmic awareness encompassing all of existence.
This relates to everyday experiences that suddenly strike us as profound. Watching something as simple as a short video can sometimes trigger a sense of freshly emerging into existence, reminiscent of Feynman's observation that "to create an apple pie, you must first invent the universe." Each experience isn't merely happening within the universe but represents the entire cosmic order converging to create that specific configuration of consciousness.
This observer gradient suggests different degrees of integration with reality—from the most basic awareness to complete cosmic comprehension. Like the scale numberline, this framework could be conceptualized as existing at every point in the universe, with each location harboring its own potential spectrum of awareness.
## The Intersection
These two conceptual frameworks share a parallel structure. Both posit that each point in the universe can be the center of its own framework. Both involve spectrums extending in opposite directions from a central reference point. Together, they suggest a universe where every location is simultaneously the center of its own physical scale and its own consciousness gradient.
This perspective resonates with various philosophical traditions while finding mathematical expression in structures like fiber bundles—where each point carries its own unique fiber representing either scales of physical reality or gradients of conscious observation.
These frameworks invite us to reconsider our place in the cosmos—not as insignificant specks, but as unique vantage points from which the entire universe, across all scales and states of awareness, can be accessed and contemplated.
Your Operating System | Eukaryotic Transcription
Phototransduction: How we see photons
How hearing works: auditory hair cells
Lecture by Douglas Hofstadter: Albert Einstein on Light; Light on Albert Einstein
"The aim of philosophy, abstractly formulated, is to understand how things in the broadest possible sense of the term hang together in the broadest possible sense of the term" — Wilfrid Sellars, an American philosopher in the 20th century.
will doesn't matter here. Only game theory. Either die by a gunshot by your commanding officer or by the enemy bullet. The second death option happens little latter if at all.