furstenheim

Parallelizing is plain simple "await Promise.all([first, second])" Problem is doing some serious synchronization like n at a time or similar. Those things are better of with callbacks

Hey, I just saw your comment.

For the jsonb `INSERT INTO some_table (a, b) SELECT x.a, x.b FROM json_to_recordset('[{"a":1,"b":"foo"},{"a":"2","b":"bar"}]') AS x(a int, b text);` That one is super flexible and it can handle thousands of records. Where you have the array it would be $1 and pass it as a parameter.

With streams it's trickier, probably you need to create a temporary table to pipe everything and then another query to insert where you want. Also you need to deal with async errors which is a pain. But you have 0 memory load. I've used it to insert/select millions of records ``` pool.connect(function(err, client, done) { var stream = client.query(copyFrom('COPY my_table FROM STDIN')); var fileStream = fs.createReadStream('some_file.tsv') fileStream.on('error', done); stream.on('error', done); stream.on('end', done); fileStream.pipe(stream); }); ```

Oh please, no auto casting. It's given way more problems that those that have solved (looking at you Excel).

If you want to use json, validate, and accept both 10 and "10" you can use json schema and do:

``` { "type": ["string", "number"]} ```

Depending on the library and language you could extend the rules to accept a string that can be parsed and throw if not.

The user does care. At a previous company we started migrating from angular to vue. Vue was so fast that those parts would be completely rendered and the rest would be white. It was also snappy.

Of course, one has to be mindful about the transition. You cannot stop business, but you can take a little extra time whenever there's need to modify a section to improve quality (both code and ui). Actually that's why we chose vue, it allowed to a progressive migration out of angular.

BTW, someone had the "great" idea of not using a framework for an internal admin portal. It turned immediately into a huge ball of mud.

There are nicer ways to do batch insert in postgres.

With 'jsob_to_recordset' there is only one parameter which is an array and it gets queried as a table. That has the advantage of avoiding different queries for different inputs (it can be prepared only once, it's harder to commit mistakes...). Also it doesn't have the overhead of creating a huge string.

Another option is copy to. Using pg copy node https://github.com/brianc/node-pg-copy-streams you can insert/query as much as you want from/to node streams without memory footprint. I've used it to pipe millions of records straight to an http response (to download a csv)

- tests in watch mode. Either with --watch if I'm doing js with mocha or inotify if I'm doing another language. Tests or compile should execute on save, which in turn I configure in autosave.

- making sure that I can execute everything from the console. It makes tickets reproducible, for my future me and my coworkers.

- go. Error handling in go has very bad press, mainly because it's boring and repetitive. But one builds the habit of handling all cases.

Writing a service thinking that you'll throw everything away is a waste of time. And so is trying to get everything perfect, because you don't yet understand the business correctly.

In my experience you should not treat all parts alike, the more foundational the more time you should dedicate.

It's important to think the db schema properly, anything else will cripple your development, and the longer in the run the harder it will be to fix it. You don't want to sanitize wrong data two years into business.

If there's a library, it's better to spend time on thinking the proper API, the code can be later be improved.

It's ok to have garbage as long as it can be isolated and you can keep on going. For example, we had configuration files that had to be synced with the db. That could have been automated, but it was ok to hardcode them in config files. It was not ok to hardcode them across the whole code. First one could be turned clean in the future, second one would've been a mess.

Invest in tests, specially setting up the process. At the beginning they can be just smoke tests (this API returns success), as the start up grows you'll have more options to add proper tests.

I think it's important to find feasible projects. A full stack project will probably be left unfinished. A way to find projects is to do tasks that you'd at work if you didn't have deadlines and were allowed to do more exploratory work.

That said. If you like nodejs, you might try to do a Babel plugin. They would allow you to do meta programming and it's an area mostly unexplored (because of the great breadth). In my case that was https://github.com/furstenheim/babel-plugin-meaningful-logs to improve error messages.

If you like java and use Intellij, give it a try at creating a plugin. You'll be able to simplify your flows. Since it's mostly self tailored it will be most probably not done. It's not extremely hard, in a couple days you might have something workable. In my case it was adding support for ZPL language, which is very niche. But most probably you can find something tailored to your dev experience.

Something that works very nicely. If the branch is always on the shape ticket-description, one can write a git hook so that all commits have the ticket name preppended to the name, like in "[ticket] commit text", very useful for understanding blames.

"however, if you want to make sure you have the same experience as your users, I'd recommend to add it to .gitignore."

That's just plainly wrong. Lock files are ignored in libraries so that shared common dependencies can be hoisted and number of dependencies reduced. That means that two users that depend on your library will get different resolutions for you dependencies according to other libraries that they use.

Geoblink | VP of Engineering, Principal Software Engineer | Madrid, Spain | Full-time | Onsite https://geoblink.com

Geoblink is a fast-growing startup that has already raised close to $8 million in investment from leading venture capital firms and was named by Bloomberg as one of the 50 most promising startups in the world! Our goal is to revolutionize how businesses in the Retail ecosystem drive their strategy utilizing Location Intelligence. We leverage GIS technologies, Big Data and advanced analytics to create a beautiful map-based user interface that not only provides lots of awesome statistics but, also a great experience for our users.

https://geoblink.workable.com/jobs/1077770 https://geoblink.workable.com/jobs/1087818