benrow

Have a look around for token biasing, or green lists. It's based on a nudge to the choice of the next token (which can always be drawn from a set of possibilities which are all probable enough).

At first I thought this approach was just the "LLM flavour" of writing, but it's way more subtle, especially as the bias is applied uniquely for each token position.

I've heard that this kind of watermarking process works by biassing the statistical sampling towards a partition of the set of possible next tokens (red set and green set), at each position. It might only be a slight nudge each time, but over a sequence of tokens, the likelihood of repeating the bias by chance is increasingly improbable.

The bias is different for each position and follows a defined RNG, seeded somehow predictably.

Can be either an open algorithm, or not. If not open, then an API could be provided to determine if text is watermarked or not.

How it applies to code - maybe it could be a subtle nudge to symbol names, etc, I'm just speculating (I only read about this in passing very recently).

You had keyboards? Luxury, back in my day we had to stamp punch cards with the tips of our fingers. If we made a mistake, the head operator would cut one of our fingers off with a bread knife. We didn't mind though, it made us tough, that and sleeping in the car park wrapped in tape from old backups just t'stay warm.

I didn't get very far into AoC this year as I ran out of time. Maybe I'll pick it up again later.

But my point is, I was surprised at how hard day 5, part 2 was. I didn't give up and solved it, but went away wondering whey I'd missed something obvious and overcomplicated it. So it brings some relief to know it was 'supposed" to be a bit challenging!

My first thought was that the pilots and crew would lose their jobs if they refused to fly.

But perhaps pilots would be less likely to get rated for Max aircraft if (hypothetically) there's a perceived broad issue with their safety? This could cause those who have a choice (stronger pilots) to go airbus.

Not saying it's the case, just pondering here.

I still prefer 200 with an empty list for the 'no results' case. The same client code works, rather than having to code for 204.

- If I query for an identifier which doesn't exist - Server replies 404, this should fall into my error handling as there's a data inconsistency

- If I query say like ?category=automotive&price=1 and I get a 200 containing a json body in which there's an empty list of matches, then my client code can handle 0 matches as well as 1 or 10 with no special handling.

I like the convention:

- 404 if resource requested by id

- 200 with empty list of results if it was a 'search' type request with params (not referring directly to an id)

I was looking for this comment. I'm very happy with the combo of IntelliJ features and vim movement commands.

The only inertia vim adds to my workflow is escaping into command mode. I have a 'jk' shortcut combo rather than escape, but if I'm hammering away I often mistime it and need to backspace out my jjkk or whatever.

We ran into a problem where we blew the direct memory limit recently.

To trace the problem, I added a breakpoint on ByteBuffer.allocateDirect() - this is how DM is normally allocated. But rather than suspending the process, we set it to print the stacktrace and continue.

This gave us handy output of possible causes. Focusing on the cases which were called from our own code (likely to be the root cause), we found a call to Files.readAllBytes() - which immediately sounds suspicious to cause a memory leak. Indeed this was the culprit, and explained why the problem was sporadic and hard to reproduce (there is some kind of per-thread caching behaviour of direct memory I never did understand properly).

Incidentally, Files.readAllBytes() uses direct memory because it uses an area of memory which both the OS can access (onto which to read the file contents) and which also Java can access (for the application code, rather than copying to the heap, which could be inefficient for large files).

Fortunately, this "by inspection" approach got us to the answer.

Others have stated profiling would work well. We'd have needed a local reproduction of the issue, which wasn't available at the time (could not reproduce without realistic workloads). Otherwise we could have used our SaaS APM software which runs in deployed envs, but that takes a sampling approach and doesn't really dig deep enough for this kind of stuff.

Google have their "Machine Unlearning" challenge to address this specific issue - removing the influence of given training data without retraining from scratch. Seems like a hard problem. https://blog.research.google/2023/06/announcing-first-machin...

Regression is a good analogy of the problem here. If you found a line of best fit for some datapoints, how would you get back the original datapoints, from the line?

Now imagine terabytes worth of datapoints, and thousands of dimensions rather than two.

Merry Christmas - Have you tried https://www.kaggle.com/datasets ?

Could still happen though unfortunately, as many bought at ultra low interest rates. Those fixes will be coming to an end in a year or two. At least in this country we tend to have fixes of 1-5 years.

Also curious.

I'm a sample of 1 and also relatively inexperienced. But I felt I quickly reached the limits of what was possible when I tried doing sentence classification with OSS sentence embedding models. The issue was with negation. I'd attributed too much magic to embedding models - they don't really understand language.

Not to say there isn't very capable tech out there. Just to add a datapoint that "sentiment analysis"-like approaches in blogs don't always scale to your particular use-case.

Edit: conscious I've drifted from the topic of chatbot type models, but felt relevant somehow.

I wrote a subtractive synth in java for my masters project. It was a fascinating domain to learn. For example, how to manage the independent voices, wavetable interpolation, envelopes. I was probably in over my head, and if I recall correctly, my supervisor was warning me about the complexity, since it was a software engineering programme.

The filter was the weirdest, least intuitive part. I ended up porting some open source code to java. This eventually worked, but was essentially impossible to debug without some sort of software oscilloscope.

That was about 18 years ago (yikes), and I still think about DSP from time to time.

The best intuition I have of low pass filters is to imagine it like an averaging, smoothing function which operates on the sample values as a sliding window. But that's not really true - it's not a straight rolling average, instead the sinc function (sin x / x) is used to scale the sample values with respect to time.

The way FIR filters work assumes a finite amount of samples (otherwise it would be technically impossible to know all past and future samples), so instead you pick a time period over which to calculate the output. Since there are always past and future samples included, this leads to a delayed output. Eg for a 100 sample filter window, you'd need to first have 100 samples in order to calculate the first output sample. This is intuitive, since a filter is sort of a smoothing, averaging function.

Here's a fairly in-depth exploration of spaced repetition, if that's the style you prefer - https://gwern.net/spaced-repetition

Furthermore "If it were measured as a country, then cybercrime — which is predicted to inflict damages totaling $6 trillion USD globally in 2021 — would be the world’s third-largest economy after the U.S. and China."

https://cybersecurityventures.com/hackerpocalypse-cybercrime...

I've also noticed over a few recent trips to Amsterdam that there's something special about the sound Ij. The river IJ, and the beer Ijwit.

The language is unfamiliar to me, and quite intriguing. I find the pronunciation of Zuid very counter-intuitive.

I wouldn't say I'm someone with knowledge (non-US, non-economist etc). But I understand the interest payments on a large national debt can be a problem. Not sure if that's a significant factor here.

I only just recently learned that national debt is sourced from bonds, purchased by individuals and companies as a safer place than cash, so take what I say with a pinch or two of salt.

A few years back I got a decent second hand student oboe for £300 on eBay. Prices are probably higher now though.

You'd definitely want to check out the second hand market if getting started in woodwind.

There should be a name for the logical fallacy where direction of travel of a statistic is taken out of context. Local fluctuation being taken as the beginning of a trend. Perhaps just "short-sightedness", but it's more specific - inference of future events based on short term signals.

It's really common in news about the economy. I guess the outcome is really important, but we only have these instantaneous signals. So something like superstition kicks in (when we're not carefully considering what is stated).

Good write up.

Sometimes novelty comes from outside the team. For example, cross-department re-platforming efforts, vendor changes, and long running migration projects. This can all lead to inconsistency, which needs to be managed so it doesn't get out of hand.

This should be factored in to a 'novelty budget' too in my opinion. A large department may have a slice of applications considered legacy-legacy, others just legacy, and some starting to move to the new stuff.

I think this is more likely to happen in companies which have been around longer - there's more accrued tech debt to deal with, there are more applications to migrate, and projects to add new functionality don't go away.

Another drawback to novelty is when keeping up to date with security vulnerabilities. If you have a common stack, then the problem is less granular.

Google have their monorepo, which (I believe) means their applications are all built against the same dependency tree, so it's simpler to keep track of upgrades.

Novelty is essential though to keep up with the innovation around us. The question is how to find the right blend of novelty (high benefit, low risk) and stability (easier to manage, maintain, and onboard engineers).

You might be interested in some podcasts by Oxide Computer Co - they talk in depth about their challenges designing and building a high speed top of rack ethernet switch.

They go into problems of high speed circuit design in general, and the high end gear and insane debugging skills needed to do so!

See their Twitter, or Spotify (Oxide and Friends) - eg "tales from the bring up lab".

At first glance, one might think Java already has collection types with defined encounter order - List (defined iteration order based on insertion order) and SortedSet/SortedMap (defined iteration order based on elements implementing the Comparable interface).

Reading more carefully, the proposal aims to lift the sequenced type higher up the hierarchy, so client code doesn't need to code to specific implementations.

So far, operations available on collection types are influenced by the implementation. So what happens if I call the proposed addFirst() on an ArrayList, rather than a LinkedList? For the former case, I doubt all the elements will get shifted up by one, so would it create a new underlying array?

(Edit: of course it would, the underlying array is recreated for resize operations already. But it would involve a slow array copy operation for what may be a common case operation). I guess my point here is, if you lift operations into a common super-type some of those operations may have implementation challenges for some subtypes.

(Edit 2: turns out it's already possible to insert an element at the head of an ArrayList using add(0, foo) so there's no change to supported methods, just an abstraction of existing APIs, as far as I can tell).

Oh, to be fair, NLP proposed the meta model, which was based on how Virginia Satir would solve people's problems by just getting them to be more specific, in a skilful way. This would highlight how our thinking can change how we experience things, through what they called deletions, generalisations and distortions.

CBT does something very similar with how it focuses on making language more realistic and less emotive.

So there are similarities in that sense.

I'd say the two fields are very different. In the case of NLP - they watched very successful therapists in the 1970s, noticed some common themes, and extracted out a model. So for example, Milton Erickson would tell artfully vague stories to his clients, which were clever metaphors for healing, tailored to the individual. It's difficult to run scientific experiments to prove the efficacy of this - it depends on the skill of the practitioner and every client will need something different, so flexibility is key, not replicability of a technique which can be codified and taught.

CBT on the other hand, definitely ticks all the evidential boxes.

I think the two can be complementary - CBT has strong evidence for success, but maybe, the codified, procedural nature of it limits it somehow.

An NLP practitioner could try something completely different outside of any rule book if the client wasn't responding to something.

Anyway, I don't have a horse in this race, just a very curious lay-person with a few favourite classic books on these subjects.

That's really interesting that neuro linguistic programming helped you so much.

I don't want to inquire into your own experiences as it's personal. But, I used to follow NLP very closely and sort of lost interest over time.

Anyway, I felt it was best suited to 'software' changes in oneself - say at level of beliefs about the world, and how our use of symbols and language can affect our experience and abilities.

ADHD, as condition seems more like a 'hardware' thing. It's intriguing to imagine how conditions like that can be helped with a 'cognitive' approach. Perhaps it's used more to help manage things through techniques and strategies.

I was in this position just recently where git suddenly stopped working, due to needing a complete update of xcode tools. Annoying blocker but OK, let's get it done.

But trying to download the xcode tools put me into a loop which wasn't completing for some reason. After several attempts waiting for it to download and install I gave up and created an alias 'git' which points to my brew install of git (in usr/local/bin I think).

This will bite me somehow very soon, I'm sure.

If I may suggest some further feedback, I think mentoring is very important at a senior level. Perhaps this could be covered in a chapter in the existing communication section, unless of course I missed something!

I think learning to identify how engineers could improve, and being able to successfully help them, is a big level-up. Departments will want to trust their seniors to do this proactively - rather than say, just when questions come up - as this helps foster a healthy team with everyone growing.

One other point comes to mind on impact - as this is often seen as a key differentiator for senior engineers. It's good to see the section on improving processes in your book, as this is a nice way to add a lot of value. Another way to make an impact is keeping a close eye on product delivery and cycle time, and identifying any problems. For example, is work getting released often, or is it getting stuck in 'nearly done'? Are there frequent blockers to builds or deployments? How is cycle time looking recently, and are there any lurking issues where work is getting blocked frequently? How do we work around this current setback - is there a short term solution to keep things moving while we work on a proper fix? Are we currently on track for delivery?

Managing complexity is another area which can be tough to navigate as a new senior.

Just some thoughts which came to mind, as we all see things from a different perspective.

Overall, I really like the focus of the book. We need more of this stuff and this will help a lot of people, so kudos on the release!

Maybe the edit buffer is a different area of memory to the mapped file. This would make sense as editing a file doesn't update it on disk until you choose to Save.