Shrinking my static site
hampton.pw
hampton.pw
Taking a simple concept like a static site and adding a ton of complex tooling around it because it is the trend now?
Why would you even need a docker image to run a static website? The best thing about a static website us you can host it everyone without requiring any extra resources, like putting it directly to some CDN as files, etc.
In production, we do exactly what you suggest: serve through a CDN.
However, for our dev environments and PR builds, programmatically setting up CDN to handle these use cases is not easily integrated with our CI/CD workflow. Our CI/CD environment is, however, good at quickly deploying containerized services.
For this, using an nginx container to serve the produced static bundle enables us to easily have PR builds for the dev team and product owners to see changes without having to check out and build the code.
I wouldn't suggest moving to docker for this use case, but if everything else was in docker, I'd want this in docker as well, even if that docker container was just running apache.
(Practically, though, if I was starting from scratch, I'd be using NginX.)
Milliseconds really don't matter for internal tools or legacy or low-use external websites. If someone wants to see our financial report from 2005, I'm sure they're happy to wait the extra 300ms to download it from a static site on the other side of the world. They should be happy that the link to it in the 2006 document they have still works. (The Apache server listens on 4 ports, to support redirecting various 2000s-era CMSs to their replacements.)
We also have some small but important sites using obscure Apache features (semantic web stuff). I have no idea if Nginx supports the necessary configuration, but since I spend 1-2 hours per year dealing with either scenario -- usually to 'archive' another site -- I'm quite happy to use Apache.
I also have this one, single site I want to be independent and not at risk of me misconfiguring caddy, playing around etc. Truly standalone.
So I generate the static code, embark a minimal web server and push it to my registry.
Took Hugo for a test drive using official quick start.
Half the themes from official docs wouldn't compile.
Started looking into architecture but official docs do not explain the big picture just a bunch of instructions.
The official documentation does not explain on what is going behind the scenes very well.
I mean, I will obviously want some sort of scaffolding and a way of changing it but this https://gohugo.io/getting-started/directory-structure/ is not helping very much.
Since you are not supposed to write go code everything has to be done through .toml config files as far as I could grok.
I feel I'd rather write my own static generator in language of my choice(say Python) than keep up with the Hugo docs.
Lately I've been writing a lot of .md when writing Jupyter Notebooks.
Thus I figured that Hugo would offer some advantages to relatives who wanted a dead simple site that they wanted to add some posts from time to time.
Static site would be perfect for them but I do not want to manually add their posts...
Started looking into Forestry.io and well it is not quite as simple as Netlify.
I thought I might have found an ally, someone who has been around for a bit and wary of the cambrian explosion of software projects. Someone who no longer believes in all that software that comes out, with new, cool features that add functionality, but, in my opinion, add subtle difficulty along the way that's hard to account for, death by a thousand cuts that must eventually lead to some kind of fatigue. Things that update and break and change and are deprecated and abandoned and require use of a different language, different build chain, different mental model, just to "add some posts from time to time" to a blog.
Please excuse this little rant. I truly believe that anyone can find their sweet spot with any system. But what is the right fit for your relatives who want it dead simple?
I've taken both routes. I have a static site that uses Hugo to generate - just like you mention. It's distribution pipeline is a Docker container inside of GitLab CI/CD. So that it has all the tools needed to both run hugo and upload to AWS S3.
Other static sites I have are generated into HTML, dumped into a docker container with nginx. Then are pulled on relevant servers which run everything else in docker as well, along with the front end load balancer. It would actually be Significantly more work to NOT have the static site in Docker.
Takes ~3 seconds to deploy a content change by typing “make deploy”
No CI platform. No docker. Much simpler. Much faster. Much cheaper than AWS.
On the one hand yes, on the other hand it's a large amount of crap just to build a static site. `npm install @11ty/eleventy` pulls 600 packages and yields a 100MB node_modules folder.
Even Sphinx (whose scope goes way beyond static site generation) "only" pulls in about 75MB worth of stuff (a third of that being Babel) over two dozen dependencies (half a dozen being sphinx's own subcomponents).
I refuse to believe that there are real people so eager to jump on the JS hate bandwagon that they unironically believed every word of an obviously satirical article.
(babel has never included a picture of Guy Fieri.)
https://github.com/babel/babel/blob/f36d07d30334f86412a9d277...
But again, people are so ready to froth at the mouth wherever JS is even mildly concerned that I really shouldn't be surprised they'd take a meme that lasted for a handful of days seriously.
Less obvious: You probably wont need Nginx, especially for small static files, in languages where the native servers are fast enough (e.g. http.FileServer)
COPY package.json .
RUN npm install
COPY package.json package-lock.json .
RUN npm ci
https://hub.docker.com/_/caddy/
You also get automatic TLS certificate management and tons of other goodies that nginx doesn't offer out of the box.
I missed the whole Caddy thing anyway because the source code was not completely open back in the days I was looking for nginx alternatives that provided with LetsEncrypt automatically.
However, it seems Caddy can be used as an alternative to Traefik if you use a plugin: https://caddyserver.com/v1/docs/docker Apparently it requires swarm though, which I don't need anyway, so, still no reason for me to try Caddy unfortunately because it does look pretty cool now.
The Caddy source code has always been open source (Apache 2.0 licensed) from day one all the way to today, and will continue to be in the future.
Beware when using other Let's Encrypt integrations. Caddy will keep your site online when other servers don't. (We saw this recently when Let's Encrypt had a revocation incident and when OCSP responders / other network infrastructure went down. Caddy kept the sites up, where other sites went down or left sysadmins scrambling to renew their certs.)
Interestingly, the LetsEncrypt incident you mentioned went completely under my radar, none of my traefik instances in production or else were affected at all, or it wasn't caught by my uptimerobot, so I'm really not sure what you were talking about then. Maybe the 2017 incident, but I was still using nginx at the time. Pretty concerning, going to check it out.
Honestly I didn't require any of the nginx paid features ever nor did I with traefik, which also has an EE edition that I just found about, I suppose needing those would be a "problem I want to have" kind of problem (meaning the project I'm taking care of is growing). I'm more into software that will generate configuration by introspecting my systems but configuration APIs are also nice to have otherwise.
Nonetheless, Caddy is a great alternative in a domain where innovation had been stalling for a while. I would like to send over a one-time donation to help Caddy development if possible to support development.
It didn't affect every site. But it did affect millions of them. You're lucky!
And in contrast to caddy, my setup actually follows the DNS and URL standards, and actually handles Absolute URIs correctly, in contrast to e.g. http://caddyserver.com./, which doesn’t even set HSTS on caddyserver.com., while even browsers consider caddyserver.com. the same origin for the purpose of TLS and HSTS.
If you’re already using something like kubernetes, which does such resolutions, this can actually become an issue. e.g. in the cluster ServiceA can call ServiceB by navigating to http://serviceb/, and if the name of serviceb overlaps with an actual real-world domain, you need to use http://serviceb./ to ensure you call the outside-world site.
So, I need all websites intended to be fetched this way to support the proper DNS RFCs.
Ever since Caddy and Traefik became a thing my tool has stopped working for more and more non-standard webpages.
Even caddyserver.com itself is broken.
In the end, I’ve simply chosen not to use caddy or traefik based websites.
how about good old ftp and a cheap shared webhost? like its been done for 30 years.
I don't know about docker and all of that, though.
It’s been a great way for me to learn, as I’ve re-written the site every 2 years or so.
You are right, this IS madness.
I'd probably choose the sftp and webhost route myself, but you know.
That's what this is doing.
You are skipping over two other parts which are installing and setting up nginx / similar and creating a process for the build itself. This ties the three together.
One of the beautiful things about a static site is it’s ability to be served by object stores like S3 as your origin, and cached by a CDN.
From an operations standpoint, you are not responsible for maintaining much of anything, the performance is super fast, highly available, and relatively cheap (no dedicated servers, just paying for bandwidth and storage costs)
Contrast that with a docker container as your origin... it must be running (and that is your problem to ensure it is).
If you’re optimizing for developer convenience, your traffic is low or not mission critical, or maybe you have a globally distributed highly available k8s cluster and that is “the way” your company does all the things... sure why not
When you manually do these things, you rely on a bunch of implicit defaults. Maybe your production server and your workstation happen to have the same version of nginx, and happen to set the same defaults. So you can test a change on your workstation and the same change works in production. But more likely, that is not the case. So you get weird differences between development and production, and you only notice when you push to production. That is not ideal. Building an image with your webserver and static files ensures that you see the same things in both places. There is no need to test anything in production, as you have a copy of the exact code and data that is going to be running in production, locally. You can tweak and poke to your heart's content, confident that you'll have the same effect when you push to production. There is no need to maintain documentation about how to build your project and what versions of things you use; you specify them in a machine-readable format and the machine dutifully builds the project correctly every single time.
(One disadvantage of clean builds, though, is that sometimes you want old artifacts to exist. Consider a case where you use webpack to generate javascript. Typically, you'll output a bundle like "main.abc123.js" which is loaded from "index.html" via a script tag. What happens when the browser loads index.html from your last build, then you the next request goes to an updated server, which says to get "main.def456.js" instead? The page silently breaks, because the server doesn't have a file called "main.def456.js" anymore. "rsync --delete" has the same problem. And if you never delete anything, you eventually use an infinite amount of disk space. So there is definitely room for improvement here, but "it will probably work if I don't think about it and cross my fingers" is not the improvement we're looking for.)
https://github.com/gunjank/docker-nginx-scratch/blob/master/...
Is there an advantage to building inside of Docker?
That's what this is doing. It creates builder containers, uses them to make the artifact and copies it in:
FROM nginx:1.17.10-alpine
RUN rm -r /usr/share/nginx/html/
COPY --from=builder /app/_site/ /usr/share/nginx/html/
EXPOSE 80
Or not working.
Naturally this lead to me gaming the system and making my docker images as in RAM usage small as possible. In the end I even abandoned SSH as too heavy and switched to shadowsocks (2MB resident) for networking the docker instances together.
He doesn't tell whether there have been any build time improvements after the changes to the Dockerfile. Will the builder docker images get cached and thus reduce the build and deployment time?
Basically the main step is the COPY . . step (previously COPY . /app) which will invalidate the cache every change to the code.
Also that on the builder, the steps beyond npm run are not needed, but well they won't improve much on the performance or space of the overall process.
But that's only if the builder image gets cached and can be reused, right?
... But in my experience it's pretty boring to wait for the whole JS rebuild when you just add a post, I think next time I'm going to remove the JS build from CI and just commit the built things when I change the frontend code, which is not often.
FROM nginx:1.17.10-alpine as npmpackages
RUN apk add --update nodejs npm
FROM node:10
RUN npm [...]For me the cause of the big image size was in
COPY --from=npmpackages /app /app
From your third Dockerfile, it seems replacing the above with the following would have done the trick without adding an extra stage
COPY --from=npmpackages /app/_site/ /usr/share/nginx/html/
Same also applies to even Java. Maven downloads tons of stuff these days. You may only be using single static string from a dependency.