How to store your app's entire state in the url
scottantipa.com
scottantipa.com
As one comparison, this is Google Finance encoding stock chart parameters:
https://www.google.com/finance/quote/F:NYSE?window=5Y
Versus Yahoo! Finance doing the same: https://finance.yahoo.com/quote/F/chart?p=F#eyJpbnRlcnZhbCI6IndlZWsiLCJwZXJpb2RpY2l0eSI6MSwidGltZVVuaXQiOm51bGwsImNhbmRsZVdpZHRoIjo0LjM0ODY1OTAwMzgzMTQxNzUsImZsaXBwZWQiOmZhbHNlLCJ2b2x1bWVVbmRlcmxheSI6dHJ1ZSwiYWRqIjp0cnVlLCJjcm9zc2hhaXIiOnRydWUsImNoYXJ0VHlwZSI6ImxpbmUiLCJleHRlbmRlZCI6ZmFsc2UsIm1hcmtldFNlc3Npb25zIjp7fSwiYWdncmVnYXRpb25UeXBlIjoib2hsYyIsImNoYXJ0U2NhbGUiOiJsaW5lYXIiLCJzdHVkaWVzIjp7IuKAjHZvbCB1bmRy4oCMIjp7InR5cGUiOiJ2b2wgdW5kciIsImlucHV0cyI6eyJpZCI6IuKAjHZvbCB1bmRy4oCMIiwiZGlzcGxheSI6IuKAjHZvbCB1bmRy4oCMIn0sIm91dHB1dHMiOnsiVXAgVm9sdW1lIjoiIzAwYjA2MSIsIkRvd24gVm9sdW1lIjoiI2ZmMzMzYSJ9LCJwYW5lbCI6ImNoYXJ0IiwicGFyYW1ldGVycyI6eyJ3aWR0aEZhY3RvciI6MC40NSwiY2hhcnROYW1lIjoiY2hhcnQifX19LCJwYW5lbHMiOnsiY2hhcnQiOnsicGVyY2VudCI6MSwiZGlzcGxheSI6IkYiLCJjaGFydE5hbWUiOiJjaGFydCIsImluZGV4IjowLCJ5QXhpcyI6eyJuYW1lIjoiY2hhcnQiLCJwb3NpdGlvbiI6bnVsbH0sInlheGlzTEhTIjpbXSwieWF4aXNSSFMiOlsiY2hhcnQiLCLigIx2b2wgdW5kcuKAjCJdfX0sInNldFNwYW4iOnsibXVsdGlwbGllciI6NSwiYmFzZSI6InllYXIiLCJwZXJpb2RpY2l0eSI6eyJwZXJpb2QiOjEsImludGVydmFsIjoid2VlayJ9fSwibGluZVdpZHRoIjoyLCJzdHJpcGVkQmFja2dyb3VuZCI6dHJ1ZSwiZXZlbnRzIjp0cnVlLCJjb2xvciI6IiMwMDgxZjIiLCJzdHJpcGVkQmFja2dyb3VkIjp0cnVlLCJldmVudE1hcCI6eyJjb3Jwb3JhdGUiOnsiZGl2cyI6dHJ1ZSwic3BsaXRzIjp0cnVlfSwic2lnRGV2Ijp7fX0sImN1c3RvbVJhbmdlIjpudWxsLCJzeW1ib2xzIjpbeyJzeW1ib2wiOiJGIiwic3ltYm9sT2JqZWN0Ijp7InN5bWJvbCI6IkYiLCJxdW90ZVR5cGUiOiJFUVVJVFkiLCJleGNoYW5nZVRpbWVab25lIjoiQW1lcmljYS9OZXdfWW9yayJ9LCJwZXJpb2RpY2l0eSI6MSwiaW50ZXJ2YWwiOiJ3ZWVrIiwidGltZVVuaXQiOm51bGwsInNldFNwYW4iOnsibXVsdGlwbGllciI6NSwiYmFzZSI6InllYXIiLCJwZXJpb2RpY2l0eSI6eyJwZXJpb2QiOjEsImludGVydmFsIjoid2VlayJ9fX1dfQ--
In both examples, the only custom parameter is setting the time window to 5 years.Dear god... this link was real. I thought it was like a joke or something
This can be overcome with html, markdown, and other rich text formats that let you specify the visible link text (missing from many chat apps*), but that's also friction to compose compared to automatically linking from just a URL.
*Slack's implementation is awesome: type the link text, highlight it, and paste a URL.
> *Slack's implementation is awesome: type the link text, highlight it, and paste a URL.
Yes! Matches my vision of hypertext perfectly: I usually type a message in normal human language and then paste links all over it for the reader who needs to dig deeper.
Most chat apps like Discord will automatically include an embed generated from a URL's metadata when present in a message.
And I believe that there is no standard API to get previews of non-public websites in a work chat, you need to develop site-specific bots or something like that?
It would be interesting if browsers / OSes / apps offered a way to use the browser's cookie jar / basic auth headers / whatever sessioning scheme in whatever apps the user decides should display authenticated previews. Ask the user to allow the release of browser data scoped to the domains that the app suggests. Similar to how some Android apps can put up a prompt to release a Chrome-saved password to an app (which I really wish more apps would include).
* http://example.com/settings/transferownership?to=efreak * http://example.com/settings/deleteaccount?confirm=true * http://example.com/sendmoney?to=efreak&amount=10000
Might need a redirect. Do previews follow redirects? Does your poorly-written chat app filter urls by protocol, or does it just look for a token with '://'? A good app will allow me to link to things like steam://connect/IP so I can invite you to a game.
* tel:9005555555 (call a phone number. Fortunately, this requires confirmation these days, but what if you're on desktop or the app that does texting can directly make calls without invoking the dialer?) * steam://exitsteam (steam will exit if it's running)
The chat host still likely needs access to the "non-public website" to discover and call its oEmbed API, so that's something you'd have to figure out, but any website anywhere can offer oEmbed metadata if it wishes.
But it's painful how few care about that!
I always clean them up before I send them (just manually - typically meaning all query parameters, e.g. with Amazon the text description after the actual ID) rarely even feel the need to check they still work, it's pretty easy, but maybe I shouldn't bother because it's only me that cares; they don't seem to notice that their links occupy half the screen and my don't!
I think it's advantageous to keep the complexity and specificity of information in URLs to a minimum outside of what's needed to retrieve the data set as it can make backwards compatibility easier - it is valuable (for long lasting tools) to have URLs that users can favorite and share for their common searches.
I agree with this sentiment. I'm not sure I've ever seen a url referred to as "good", I was just a bit confused as to why someone would baulk at a URL. In my opinion everything after the host portion of the url is to serve the apps needs, so it knows where to go or what to do, it's not really an interface for the user as obviously there are better ways to present that.
* Should be book-markable.
* If searching with a query, should have a clear query parameter for ease of browser integration.
* If the URL contains slash dividers that can be read as categories (e.g. storefront.com/department/category/product), then deleting everything after a given slash should go to that category’s page.
* The link should be short enough to be sent over a plain-text conversation without becoming a wall of text.
* The link should be short enough that it cannot hide a malicious subdomain (e.g. Google.com.flights.malicio.us/more/elements).
Opaque slugs are problematic in a low-trust world. Phishing attempts are easier when the parameters are indecipherable. I'm not clicking on that shit in a text message or an email. I've seen what happens to people when they do.
If the URL has a bunch of goop in it, people are going to make fun of me for it, especially if it's super long and doesn't fit in a single line. Or if there's stuff in there that's specific to me and not the thing I want to show them.
It's awesome and incredibly handy.
Yahoo is on the ball here.
I think they are. If I want to go back to some page I was looking at earlier, it's really nice to be able to find that page in my history. That Yahoo URL is going to be hard to differentiate if you looked at a few stocks.
It's also handy if the URL's are predictable enough that I can hit one directly rather than having to go through search. Good for me because I don't wait on page load, good for them because they don't have to serve my search request.
It's pretty low on my list of UX annoyances, but it is there.
What if they encrypted the parameters with a one-time pad ?
Clarification, the human user probably won't want to modify the URL, but the client will.
{"interval":"week","periodicity":1,"timeUnit":null,"candleWidth":4.3486590038314175,"flipped":false,"volumeUnderlay":true,"adj":true,"crosshair":true,"chartType":"line","extended":false,"marketSessions":{},"aggregationType":"ohlc","chartScale":"linear","studies":{" vol undr ":{"type":"vol undr","inputs":{"id":" vol undr ","display":" vol undr "},"outputs":{"Up Volume":"#00b061","Down Volume":"#ff333a"},"panel":"chart","parameters":{"widthFactor":0.45,"chartName":"chart"}}},"panels":{"chart":{"percent":1,"display":"F","chartName":"chart","index":0,"yAxis":{"name":"chart","position":null},"yaxisLHS":[],"yaxisRHS":["chart"," vol undr "]}},"setSpan":{"multiplier":5,"base":"year","periodicity":{"period":1,"interval":"week"}},"lineWidth":2,"stripedBackground":true,"events":true,"color":"#0081f2","stripedBackgroud":true,"eventMap":{"corporate":{"divs":true,"splits":true},"sigDev":{}},"customRange":null,"symbols":[{"symbol":"F","symbolObject":{"symbol":"F","quoteType":"EQUITY","exchangeTimeZone":"America/New_York"},"periodicity":1,"interval":"week","timeUnit":null,"setSpan":{"multiplier":5,"base":"year","periodicity":{"period":1,"interval":"week"}}}]}
I bet if they were more aggressive in using defaults it would be a lot smaller.
https://www.ovh.com/fr/order/webcloud/?#/webCloud/domain/options?selection=~(domains~(list~(~(name~'somethinghn.com))~options~(~'dnssec)))Probably written in Common Lisp? Based on some quick searching I found this reference https://docs.ovh.com/us/en/web-paas/languages-lisp/
I suspect the main reason programmers dislike Lisp syntax is that they try or see an example of some basic arithmetics, get surprised by the prefix notation that's so "unnatural" compared to the "obvious" infix notation they learned in primary school, and write off the entire language as weird, not noticing the fact that infix math is an exception in programming languages, as otherwise almost all code they write in any other language is in prefix notation too, just with a parenthesis shifted to the left.
A somewhat mind-blowing observation to some people:
(foo (bar baz quux))
foo(bar(baz, quux));
The two lines above are both in prefix notation. You can go from the first to the second one by a) shifting each left paren by one token to the right, and b) adding commas between whitespace-separated tokens. (defn fib [x, n]
(if (< (count x) n)
(fib (conj x (+ (last x) (nth x (- (count x) 2)))) n)
x))There was a programming language in the 1970's called SAM76 that didn't care how many closing parentheses you had, as long as it was more than the number of opening parentheses.
So, something like this was perfectly valid:
a(1+r(em(2)))))))))))))))))))It screams "trust me all good with the closing parantheses".
On the other hand, auto closing parentheses in code is a terrible idea for obvious reasons. There are only like 7 languages used in industry and they all have excellent tooling that integrates with your editor to eliminate issues like this.
if ((v1.x()-v2.x())*(v1.x()-v2.x())+(v1.y()-v2.y())*(v1.y())-v2.y())+(v1.z()-v2.z())*(v1.z()-v2.z()) > distance*distance) {
// collision
}
Would you like this to fail to compile? Or would you prefer it to silently return the wrong value?where you have defined an actually readable equivalent inside of collisioncheck would be my prerogative... I also agree paren matching failures should error, but your example is NOT why.
SomeUrlService.pushState(JSON.stringify({ foo: 'bar' }));
SomeUrlService.navigate('/some/url/here')
Likely, that "SomeUrlService" is also tacking on plenty of other shit to the URL that is passed along. I don't work for Yahoo and never have, so this is all pure speculation.
Absence of special characters. No need to escape symbols like {}:/\?& etc, etc.
H4sIAAAAAAACA7VU3W7TMBS+5ykmcxuNdOlGyd3QmECCIdYWNKYKnSanqVfHNrbTNqoq7QF4Sp6EYyfpaMYtdz7H5/vOz3fsHePSoVmDYCnbIK5YxDQarnKecVezdBAxx0ucSu5YKishIpaBzAV+47lbsnR4mgxHF+dv4jgZJYPh4PV5xBaCa405SxcgLEZsrUTlKXI0AojTmYq8kD90x8woa5fAzcGxBOMmtUaqSnCJVBVuHRLDE2sJZoVujNZyJS1Ld3viLAqDBTjytGi1FBlrCccZiI4RDHmtq3KOHst+P/6iMk8qmRs6eo9rCDovhXOpKxeiOZXRh0Qs51aHBvtXVJmqXAee6pOvYSIU+DKO5/HFgMBXaiP/8i8WSZKAR2qQ6NUJLXh5wECJJFog23gZriFzioYXnw7P215vIPA0oP2+4wmYxkkHUjpD6YLKT8Vfs39x+PZz3FKSiNWXWx6Y5HGEVpb72TebQjlroMCP78csvZ+11m2wDpDepGa+UkuyUrU+QVkJx7XgSN1Ra3OwPl/dyHe0qLvWDM30l9qzetXbpT3z0htOO/oWslVhFKXvVg/XNBF7WEQl/GC9TqPB4ow9Ax7jPoEOA1ZGKwMOvZHz9YGORsw7ct8oL65w7VeXjKyyTpW3IAvsXpqty7nymt3v2nMrT2N8nj9gFoQ8uvxZKYft+r/7Mv0wuQvPhyZO1BN6zN+V9HeXJQ0sg1c3uPlxp8wqLFvv7T//Hfq/wf8Qaz/bv/gDbvc6N5oEAAA=
In theoretical terms (in particular Kolmogorov-complexity perspective), I believe the inclusion of a dictionary wouldn't help, it would strictly increase size. It can help (specially from a Shannon information perspective) if the compression is too computationally demanding to build the data model (which is kind of what all compressors do); but it almost certainly doesn't help in the case of something as small as an URL. But then it's possible to just pack a common compressor with the website (and just refer to it with a version code).
I think the web in general would benefit greatly from some kind of standard compression libraries. Compression tends to have a significant cold start cost to work well, if that were mitigated certainly something like an url could be far shorter.
I believe HTTP2 has header compression (RFC 7541)[1] with 'static tables', which seems to be a form of dictionary, but surprisingly to me no html compression with such shared dictionaries. There is caching libraries (unfortunately cross-domain caching seems to be deprecated[2] due to security concerns) I believe which helps in other ways, but I think a true dictionary-style compressor would bring huge bandwidth savings.
Major web entities should get together to develop those standard dictionaries (and which algorithms to use them with), so they are reasonably fair for everyone (maybe even language specific dictionaries should be used).
Security seems indeed a concern, but I think the key would be carefully preventing that a content were decoded with the incorrect dictionary (meaning the server could serve a different web page than the standard decompression) -- but overall it doesn't seem like a big issue (similar to cache, less problematic than cross-domain caching).
In the future maybe even one of those fancy neural network (or otherwise machine learning inspired) methods could be an option, specially useful for highly bandwidth constrained like rural or satellite internet (although of course performance is always a priority with mobile devices being major users).
[1] https://httpwg.org/specs/rfc7541.html#static.table.definitio...
[2] https://www.stefanjudis.com/notes/say-goodbye-to-resource-ca... Is this still accurate?
Correction: standard compression dictionaries
To my understanding, that static table in HTTP2 is directly in context of recommending and discussing Brotli compression, which does use a table like that as a standard static dictionary in HTTP2+ scenarios including other static dictionary inclusions derived from a corpus of HTML documents.
> To my understanding, that static table in HTTP2 is directly in context of recommending and discussing Brotli compression
It seems separate, that would be a header compression for HTTP2 header itself, while Brotli encodes content (usually html, css or js).
[^] this statement is a bit of a lie for reasons of simplification
https://stackoverflow.com/questions/417142/what-is-the-maxim...
As a native speaker of en-US, I would write this as "base64 does not have the problem here, iMessage does."
Your version reads more correct, thanks for pointing it out.
It seems like storing state in the URL could benefit from URL shortening techniques so long as the secret sauce logic is both client and server side.
it additionally gzip's the data to compress it futher, with.. some savings..
in the above example you've shared, gzip would've,
orig=1177
base64=1572
gzip(orig)|base64=768If it points to a cloud database then you need a cloud database to do what a url could do.
[1]: https://www.kayak.com/flights/SEA-IND/2023-01-10/2023-01-14?...
in general i've been forming a thesis [0] that there is a webapp hierarchy of state that goes something like:
1. component state (ephemeral, lasts component lifecycle)
2. temp app state (in-memory, lasts the session)
3. durable app state (persistent, whether localstorage or indexeddb or whatever)
4. sharable app state (url)
5. individual user data (private)
6. team user data (shared)
7. global data (public)
and CRUD for all these states should be as easy to step down/up as much as possible with minimal api changes as possible (probably a hard boundary between 4/5 for authz). this makes feature development go a lot faster as you lower the cost of figuring out the right level of abstraction for a feature
0: relatedly, see The 5 Types of React Application State, another post that did well on HN https://twitter.com/swyx/status/1351028248759726088
btw my personal site makes fun use of it -> any 404 page takes the 404'ed slug and offers it as a URL param that fills out the search bar for you to find the link that was broken, see https://www.swyx.io/learn%20in%20public
1: https://developer.mozilla.org/en-US/docs/Web/API/Window/sess...
Still an underappreciated browser feature though, esp for class server-side rendered apps.
ive never honestly used sessionStorage as it's not intuitive to me (if i want it clientside i should probably track it serverside, and at that point server is my source of truth)
It's not that you shouldn't say "there be dragons" or suggest existing solutions, but that's doable without saying you shouldn't do X at all...
Also, if you are careful you get undo and redo for free with the browser's back button doing all the work for you.
The disadvantages are that your representation of internal state becomes part of the interface - if you ever change your app you need to deal with versioning the state so your new version can transparently handle the old state format.
If your app has a server component that acts on this state, be super careful about acting on it and treat it as you would any other input under user control.
If you app is completely client side, consider storing the state in the #fragment section of the URL. This never gets sent to the server. An example from my own site [0] - see how the fragment part of the URL changes as you select different topics.
There are also limits on just how much you can cram into a URL but with care you can shove a lot of state.
I would recommend signing it if it's generated by the server component, and checking the signature when the server component is provided this signed state.
For example to do this in Node is quite straightforward.
Key generation:
const crypto = require('crypto');
crypto.generateKeyPair('ed25519', (e, pubkey, privkey) => {
// save pubkey and privkey somewhere
// ...
}
Signing: const data = Buffer.from(JSON.stringify(state));
const signature = crypto.sign(null, data, privkey);
const signeddata = `${data.toString('base64')}.${signature.toString('base64')}`.replace(/=/g,'');
Verification: const parts = signeddata.split('.');
const data = Buffer.from(parts[0], 'base64');
const signature = Buffer.from(parts[1], 'base64');
if (crypto.verify(null, data, pubkey, signature)) {
// signature not verified, throw or return
// ...
}
const state = JSON.parse(data);
As the above uses Ed25519 the signatures are quite small too. It needs a bit more error checking, and might need extras like expiry time and such, but should be roughly sufficient.But if your state is so important it is probably better to not uses these techniques at all and just store the state on the server.
If you sent the client state to the server for some type of CRUD action or persistence, you’d need to sanitize and validate it first. And at that point like you said, why not just keep the state secure server-side and not trust the client.
This is the biggest reason to avoid this. URLs aren't meant to be used this way. I'd only do it if I wanted a quick and dirty solution.
I disagree, URLs are supposed to point to a resource - in this case the resource is the client state of the app (aka "deep linking"). It is a useful approach.
Of course, it can easily be misused.
For example, you may track the scroll position for the purpose of loading more content when the user reaches the bottom of a feed, but you don't want to persist that across reloads. And the user may click on some content and anchor the view to it, which you do persist in the URL for the purpose of history or sharing.
Conversely, you can't link to the newest version.
Also, consider whether you want to allow the user to share the URI or not. If not, then you need to always check that the user is authenticated as whatever you wrote in the state cookie.
Ugh! Please don't do this!
(1) https://support.microsoft.com/en-us/topic/maximum-url-length...
My reading is you were worried about length of encoding one state, so you moved to encapsulating states in a dictionary with keys of hash of State and objects of State
And this led to a decrease in size of the URL?
My guess at my misunderstanding: you kept the state dictionary server-side or at least some other storage method than the URL, and just used the URL to identify which state to use from the dictionary. I e. The bit you add to the URL is just the hash of the state, or dictionary key
IE was all about limitations Nothing surprised me about IE limitations.
https://github.com/Cyphrme/URLFormJS
I use it in a lot of places, like: https://convert.zamicol.com/#?inAlph=Text&in=Hello%20world!&...
https://cyphr.me/ed25519_applet/ed.html#?msg=Hello%20World!&...
https://cyphr.me/coze#?verify&input={%22pay%22:{%22root%22:%...
You'd need to read state off the url fragment (location.hash). You can feed that into your appropriate state management system, I suppose. Some (like react router) I think support hash routing, but I don't know that they support both has routing and history routing at the same time, so that may not be quite up to task if thats the case.
Still though I suppose you could probably gzip it before base64 encoding it for some additional optimization.
Extremely long URLs have other UX issues, e.g. sending them on chat apps and having them eat up multiple scroll pages in one URL, like:
"Check out this event:
http://..... ... ... (500 scroll-screen-lengths of just URL) ... ...
Here's another event on the same day:
http://..... ... ... (500 scroll-screen-lengths of just URL) ... ...
Can you let me know what you think and which one you'd like to go to?"
A decade-plus ago, this was common in single-page apps. It was the only way to do it before the history API existed, and enough people were doing it that Google was pushing a fragment-url thing [0] that allowed server-side rendering. You'd make such links in a certain way, then the crawler would hope a simple modification of the URL could be rendered by the server and load that.
(Per a comment on that source, it's not been a thing since 2015, since the google crawler can now handle javascript)
[0] https://stackoverflow.com/questions/6001433/does-google-igno...
https://github.com/Cyphrme/URLFormJS
I would love to see it get more use.
Here's a small demo: https://cyphrme.github.io/URLFormJS/#?first_name=Bob&last_na...
See my other comment on this page for some other examples of its use.
There's a larger issue of URL fragment delimiters. For a practical work around on Chrome only, fragment directive/text fragment can be used as a delimiter. (In my opinion, Chrome's delimiter should be re-worked with a new, more comprehensive delimiter.) Example of the workaround: https://en.wikipedia.org/wiki/URI_fragment#References:~:text...
I find it interesting to sometimes play around with the pasted data just to see how it compresses. Sometimes adding a character or two may cut several characters from the URL length!
And then when I think about biology/genetics and what kind of unfathomably weird and convoluted compression is going on there, and then think about whatever it is our brain is doing… and then when I think about how like all of our perception is probably some kind of weird inescapable compressed representation of true reality…
It’s all magic. Real life magic.
The entry point to the compression is the syncCompress[2] function, which converts the data to UTF-8, adds the needed header bytes, and then compresses using brotli-wasm. Decompression is done in a streaming way in streamDecompress[3]. This is to avoid zip bomb attacks, where even a short URL could decompress to gigabytes of data, locking the browser. Thankfully brotli-wasm had streaming decompression builtin, I just had to write the wrapper code to update the text content and the status bar in the UI.
You can find the brotli-wasm code I'm using in the vendor folder[4], there is a JS wrapper and then the WASM code (both are straight from brotli-wasm).
[1] https://github.com/httptoolkit/brotli-wasm
[2] https://gitlab.com/Nicd/t/-/blob/05e587b0183ff80b1c6e050b5d3...
[3] https://gitlab.com/Nicd/t/-/blob/05e587b0183ff80b1c6e050b5d3...
[4] https://gitlab.com/Nicd/t/-/tree/05e587b0183ff80b1c6e050b5d3...
FYI, this is almost entirely untrue. Maybe for some old versions of IE, but modern browsers can handle extremely long URIs
const [myState, setMyState] = useState(null);
I've created at least 3 libraries that follow that same pattern, but instead of ephemeral the state is saved somewhere: // ?firstname=
const [firstName, setFirstName] = useQuery('firstname');
// localStorage
const [name, setName] = useStorage('name');
// global state (that can be connected to other places)
const [user, setUser] = useStore('user');Also, another shout out to mermaidjs and flowchart.fun for also implementing similar url-based sharing.
Still on FF I opted to use LZ Compression (pretty sure many other sites do this as well) to get the number of characters down
I haven't used https://github.com/ipfs/js-ipfs in this capacity but I'm under the impression that that's moving bits around like that is more or less its purpose.
Although I suppose this puts a burden on the URL-creator to pin the content until the URL-clicker doesn't need it anymore, which is not how URLs are supposed to work.
Wrote this SerDe which can be extended with better algorithms in the future. https://github.com/mermaid-js/mermaid-live-editor/blob/devel...
Eg: https://mermaid.live/edit#pako:eNo9kUGTnCAQhf8K1adNlesoqKiHV...
Here is the compression code: https://gist.github.com/kissgyorgy/39623a7d0ba2f6faafb464b72...
I used the properties of my data structure (multiple cards represented in a byte as bits) to represent the same data on less bytes (to avoid compressing JSON with unnecessary characters)
Bad for ordinary, mundane web applications and sharable URLs!
Like, great for a musical step sequencer in the browser, bad for a website or HTTP service
OTOH, app state on websites of commercial service prodivders might be best represented by a JSON blob anyway.
In something like a checkout process or form submit, of course this doesn't matter much.
But I prefer human-readable URLs wherever possible.
And I implemented a similar pattern once, too.
It's a very interesting approach in intself.
Relevant POJO state tends to be small enough but long lists or resourcrs in a URL are not supported by most servers and not advisable.
Good for small scale unimportant external state management but not for your everyday web app.
We’re so close/all the pieces needed seem to exist, but getting something like that off the ground is super difficult.
If you don’t want to store anything on the server, why serve the static JS and HTML? You can embed them in the URL also.
In that cass the URL just becomes a local file.
You should store the file locally, along with all the data it embeds.
Or you can store the files on a CDN at the edge. The problem with the latter is that people can abuse the storage.
Really, the Web needs to be overhauled to have a topology more like a DHT, where you only have peering agreements with a few peers, and they cache the content temporarily when it is requested a lot. So no centralized servers at all.
https://sfc.vuejs.org/#eNo9j71uwzAMhF+F5eIWqCV0NZQA3foGXbikj...
It’s the same as base64 but w/o "+" and "?" symbols.
This is so valuable that it can't be overstated. Being able to store the raw data in a link WITH the visual representation, you'll never struggle trying to find out who has the latest version.
I saw someone forked it and removed the lodash dependency / minor refactoring https://github.com/ananthakumaran/u/compare/master...j-te:u:...
The purpose in this case was to generate a QR code, the idea is this can be included in physical payslips. Then the numbers behind the calculations (tax deductions) can be validated, broken down, and understood. I'm developing more tools to these ends, via my overarching project calculang, a language for calculations [1].
I also have a loan/repayment validator [2] but haven't added this QR code feature yet.
Bank letters e.g. "Interest rates are rising and now we want 100e more per month, every month" could use a QR code to an independent validator or to see the workings behind the 100 calculation.
Not using this in the real world now and there are security considerations to keep in mind, but reading state from a URL facilitates the usecase: QR codes that link physical numbers to their calculation and model.
Implementation of payroll calculator is an Observable notebook and thankfully it neatly supports all my strict requirements as demo of this.
[0] https://observablehq.com/@declann/payroll-playground-ireland...
+Feature tweet: https://twitter.com/calculang/status/1608183731533107206
Back before websites were "Apps", you could often do exactly this (Including Authentication!) right from the url.
If they forgot to include the session token on any link on the page, you'd be logged out and have to authenticate again, or you could navigate backwards back to a sessioned address in your history. Once I figured that out, I copy and pasted my session id so if they omitted the session on a link I wouldn't have to log in again.
Perhaps instead, assuming the user is online, hash the json state data, then send the json+hash to the server and update the URL to URL?hash. Then the user has a much shorter URL, and the server can lookup the json state from the provided hash to resume the app at that state. As a bonus you then have versions of the data over time (either posted automatically in the background or on-demand when the user clicks Save).
The idea is that instead of a base64 string, you get something that is much easier to edit in the url bar:
> name=John%20Doe&age=42&address.street=123%20Main%20Street&address.city=Anytown&address.state=CA&address.phoneNumbers.0=123-456-7890&address.phoneNumbers.1=234-567-8901
There's a similar npm project to [0] that I discovered after I'd published this, but I don't recall the name. PRs welcome.
The native URLSearchParams browser interface even supports this with the getAll method.
https://developer.mozilla.org/en-US/docs/Web/API/URLSearchPa...
The “genius dev lead who went to Princeton” kept pushing that they went to Princeton so people stopped arguing against it
Tried to do this securely and got pwned in the very first demo
You’re technically correct/sounds like you agree it’s still a bad idea, even if there’s a way to duplicate certain state for validation and make it work. Staying shackled to the earth via stupid rules of thumb like minimizing client state in banking apps does keep you from flying straight into the sun, though, which is nice.
...made in 2013 :) Here's the js that runs it: https://max.io/bash.js
Suppose that you have a hierarchy that was opened. Do you want to store all the hierarchy breadcrumbs in case you hit refresh, and reload all of them?
Furthermore, what about URLs to initiate an action from a particular context? If you refresh what should it show? It should show a dialog / lightbox above the page, showing the form / interface for filling out before taking this action.
Here is a question for you: what happens when you have a url for a New Issue in github? By itself. It renders a form for a new issue full-page. But now what if I want it as an overlay over a hierarchical state that U navigated to? I should have a longer URL and it wouldnt be full screen anymore.
The main route of the url (example.com/page/subpage) represents the background page of the overlay.
You have two options. 1. Using further subroutes to represent the stage of the overlay if your framework supports it (Ember does) (example.com/page/subpage/form/step1). 2. Using query parans to store the state of the overlay (without the actual data of course): … subpage?overlay=form&step=1
Also breadcrumbs should always mirror the routes. Frameworks, libraries can automatically do it for you.
https://en.wikipedia.org/wiki/WebObjects
Was quite revolutionary at the time and it also included the first (or one of the very earliest at least) SQL ORMs.
I do this in all my apps. https://ohayo.computer/, https://simoji.pub/, https://try.scroll.pub/, https://jtree.treenotation.org/designer/ etc.
At one point I also made a tiny lib to make it easier:
When I first came up with this idea, I was going to write some sort of server backend rather than just doing everything in the frontend, but once I started actually working on it, I realized that it wouldn't really add any value and it would make it a bit more annoying for people to self-host. Since then, I've wondered a bit how viable "static pages with state in the URL that can be easily self-hosted" would be as an alternative to a desktop GUI or Electron app for more technical users. One of the nicest parts of this approach to me is that I was able to make in only a few hours one evening despite knowing only the basics of webdev from maybe 8-10 years ago and next to nothing about programming GUIs in general. This makes me think that it wouldn't be too hard for people to fork and modify it to their own liking (e.g. changing the hard-coded style settings for rendering the code snippets), so there could be potential for an ecosystem to grow organically around something like this.
In case anyone is curious to take a look: https://gitlab.com/saghm/pastml/
We use it in Lighthouse for the exact same purpose, a URL #hash full of state. We found that modern browsers support 10mb of data in the hash. https://github.com/GoogleChrome/lighthouse/pull/12509#discus...
I can think of many apps that don't need server side data storage or security that would benefit from this type of state saving however. For example, this app that I was using today - https://wheelofnames.com/ . Right now to save I need to create an account, presumably to tie my account to the saved state on the server side. That's pretty heavy/intrusive when all I want to do is keep the names list and preferences populated for my next visit.
https://github.com/recurrency/urltron
Perfect use case for storing page state in url.
We did this once with a web-based REPL, where you could click a button to copy a link to something you'd previously eval'd and then send that to someone else and their buffer would be preloaded with it. It was super nifty, until someone tried it with a buffer that was a few hundred lines long and at least one browser just said "nope". It sounds like OP is doing some smart stuff to minimize URL length, which should help, but there's still a limit somewhere
The theory was wonderful (yay! stateless load balancer etc.) but in practice browsers at the time were less than happy with trying to store that much state in the URL.
I didn't even know about window location hash, which I just learned from the other comments. Thank y'all.
https://github.com/jncraton/box-line-text
The data representation could probably improve in terms of readability, but I'm happy with the way it provides a concise encoding while still including the text from the document directly:
https://box-line-text.netlify.app/#;c2121Hello;5321;c1341Hac...!
However as some people have pointed out it comes with a few caveats
- The URL is unreadable for humans
- You possibly end up storing things in the URL you did not intend
- For smaller data sets the URL actually becomes larger than the URL params equivalent
- You end up having to deal with version issues, possibly even URL string migrations as the code evolves
I'm sure there still are use cases where this makes sense, but for most applications this ends up being overly complex
https://www.typescriptlang.org/play
I'm mixed on it. It means it's mostly useless for having your own list of creations because you have to manually store the URLs (PITA) and you need to store them somewhere accessible if you want to repo the UX of the site actually storing them where you can more easily access them from anywhere.
The rationale being that long URLs are often suspicious and could potentially be used for SQL injection or path traversal attacks. Whether or not this is a good heuristic is left as an exercise to the reader.
https://stackoverflow.com/a/417184/768795
I wouldn't consider state in the URL fully general, as data can easily exceed that for many apps; but it's a useable technique where it fits.
Meaningful individual parameters are preferable, but similar size limits apply either way.
I know of at least one bank where this was exploited, because people see the base url, assume safe, and can't read the embedded escape / javascript in the longer part. Especially when the link is zipped into a little clickable thing and you can't see the URL at all.
Since the OP is using lz-string to do compression, I suppose the data should be repetitive.
The reason I did the compression (using pako as well) was that without it the URL was too long to show a preview on iMessage. I also compress the state manually myself which saved a bunch of bytes too.
It's more likely that messaging app will have stricter limit than Chrome (and other browsers at this point do not matter).
Though I'd suggest to use some kind of server storage if you need to transfer more than few kilobytes of data.
But yeah, the length will get long and even if the browser can handle it do you really want to be passing around that much text at once?
I'm also curious what various OS clipboard size limits are.
https://stackoverflow.com/questions/1374753/passing-base64-e...
Just wondering never created an entire app around this idea
As some comments suggest, I will move the data from the query string to the URI fragment to avoid the length limitation.
Which is totally missing the point and being intentionally daft but it is a little amusing XD
Show HN: URL Snake - https://news.ycombinator.com/item?id=34288577 - Jan 2023 (30 comments)
For example: https://themeasureofaplan.com/market-timing/#initialContribu...
This shows the performance of a market timing strategy on the S&P500, where you invest $20,000 upfront and $1,000 per month thereafter, while only buying when the market is 25%+ away from the all-time high price (analysis from 1993 to 2023).
These user input variables are all directly encoded in the URL, after the # symbol.
They both achieve the purposes of allowing custom URLs that bring the user to a specific state of the app.
Edit: better link to explain the concept
* back/forward button behavior
* cache behavior
* bookmarks over time
surprising there's no discussion of it.
Just wondering, how long did it take you to put the app together?
While technically URLs have no formal length limit, various sources suggest that URLs with more than around 2,048 characters start causing issues in browsers, and around 8,196 start causing issues in CDNs (https://stackoverflow.com/a/417184). You can work around this partially by not storing this state information in the query string (path/to?<state>) and instead using a hash string (path/to#<state>), and then extract that data in JavaScript such that it's not sent to the server at all.
IMO, the canonical way to solve this problem is using protocol buffers, which you can then serialize into a much smaller number of bytes, which is then base64 encoded. For example, as mentioned in another comment in this thread (https://news.ycombinator.com/item?id=34314578), Yahoo Finance has a 1,616 character-long URL with 1,572 characters of base64-encoded JSON state:
> {"interval":"week","periodicity":1,"timeUnit":null,"candleWidth":4.3486590038314175,"flipped":false,"volumeUnderlay":true,"adj":true,"crosshair":true,"chartType":"line","extended":false,"marketSessions":{},"aggregationType":"ohlc","chartScale":"linear","studies":{" vol undr ":{"type":"vol undr","inputs":{"id":" vol undr ","display":" vol undr "},"outputs":{"Up Volume":"#00b061","Down Volume":"#ff333a"},"panel":"chart","parameters":{"widthFactor":0.45,"chartName":"chart"}}},"panels":{"chart":{"percent":1,"display":"F","chartName":"chart","index":0,"yAxis":{"name":"chart","position":null},"yaxisLHS":[],"yaxisRHS":["chart"," vol undr "]}},"setSpan":{"multiplier":5,"base":"year","periodicity":{"period":1,"interval":"week"}},"lineWidth":2,"stripedBackground":true,"events":true,"color":"#0081f2","stripedBackgroud":true,"eventMap":{"corporate":{"divs":true,"splits":true},"sigDev":{}},"customRange":null,"symbols":[{"symbol":"F","symbolObject":{"symbol":"F","quoteType":"EQUITY","exchangeTimeZone":"America/New_York"},"periodicity":1,"interval":"week","timeUnit":null,"setSpan":{"multiplier":5,"base":"year","periodicity":{"period":1,"interval":"week"}}}]}
The un-base64'd JSON dictionary is 1,162 characters long, 569 of which is composed of just the key names (48 percent!), and most of the remaining fields likely contain their default value. If this was instead encoded in protobuf, the key names wouldn't need to be included, since fields are referenced by their field ID, and default field values would take up little-to-no space since when the default value is present it isn't encoded into the structure. I presume that, if done efficiently, you could likely make an equivalent representation in protobuf that is 1/4th or even 1/8th the size. You also get, for better or for worse, what is essentially obfuscation for free (yes, you can read out the packed format, but without an equivalent protobuf struct definition it's still hard to tell what field is which, and the entire operation becomes much more labor-intensive).
i used sveltekit and vercel
A pet peeve of mine is that webapps shouldn’t hijack the browsers default functionality, here being back/forward that should refer to pages, not mutations of the same page.
Ideally, one would have a separate undo/redo button on the page for page or url mutations (which should always be in sync). But mutating the URL shouldn’t be done by pushing to the browser history stack, or if pushing one ought to remember to pop the current entry first so the result is a single mutated page on the history stack.
ClickHouse Playground: https://play.clickhouse.com/play?user=play#U0VMRUNUIDE= ClickHouse Dashboards: https://sql.clickhouse.com/dashboard#eyJob3N0IjoiaHR0cHM6Ly9...
The second example shows you why it is not an entirely good idea. The second link cannot be posted in Slack and Google Meet, so I have to use an URL shortener...
Luckily I have it in my disposal: https://pastila.nl/ Example: https://pastila.nl/?02a0d469/8f49f7e4406bd4fbc0e7b142c470dfb...
Naturally, the 16 Megapixel images are sent as an array of json floats…
Extra fun fact: Two years into the project, they hadn't heard of REST. Then rejected it, because, I quote, "The S means stateless and [they] need the server to store state"
Elaborate
> I also didn't want to store anything on the backend (at least for the free tier).
You can make the state unique in the DB by a user cookie ID and then upsert the state which will overwrite older state with the latest state. Then you only have 1 record per free user. Remove record that haven’t been modified in 30 days and you don’t have much overhead. Just off the cuff thinking.
Especially as in the second part of your comment you restrict shareability/usability significantly from what the author wants to achieve (IE I can't as a free user bash out a diagram and share it, then later do another and share with a different person).
In fact the solution is so elegant when you think on it; free account volume is incredibly scalable, zero storage cost, offer immediate value but also a really simple upsell to pro accounts (save to the DB so you don't have to save the URls).
no it doesn't. If my URL is `/thing#a1234` and I share it with you it will load my state for you. If you make a change, it's easy to push a new hash to the url that belongs to you.
> I am genuinely intrigued as to why that's better?
I didn't say it was better. We engineers are supposed to discuss solutions. I ended my last comment with "Just off the cuff thinking".
> Seems like it could be better
Maybe you didn't mean it that way, but it does come across that way. Just a data point for you; I got no beef...