HNHacker News
TopNewBestAskShowJobs

elmolino89

189 karma · joined September 27, 2018

submissionscomments
elmolino89··on An Excel error led Austria's SPÖ to announce the wrong candidate as the winner
In any Excel sheet everything is mixed up: you got raw data, calculated values, formulas, formatting, plots etc.

With a Jupyter notebook you have these well separated, and there is a little chance that someone somehow changed the font in one of the 20k cells with values. Scaling up plots or changing the color scheme are explicit: you must have a cell changing the defaults. Yu can not just muse over some cell, press Del and plonk it. If you verified say md5 checksum for the raw data file it contains what you expect it to contain.

If you follow up actions of an average Excel user, then it is a long chain of mostly purely manual manipulations, like selecting the cells, copy and paste, add color here and there etc. A lot of steps to check and verify. Compared to SQL or data frame manipulations it is too complicated to be really useful IMHO

elmolino89··on An Excel error led Austria's SPÖ to announce the wrong candidate as the winner
IMHO instead of ramming a square peg into a round hole one may think of switching from Excel to i.e. Jupyter notebook(s). Keep the raw data separate from the calculations/visualizations and get a more trackable environment.

You may also pinch out bunch of functions as separate scripts in a git repo and share these between various notebooks/people.

Calculate md5 checksums of raw data files. If values in rows can be modified, get the hash sums (xxhash?) per row. That way even with the lowest common denominator (text CSVs) you know if the inputs changed and where.

For larger data consider storing it in parquet format and using duckdb.

Assuming that you must have Excel output calculate what's needed using raw data, notebooks/scripts then output Excel using openpyxl

elmolino89··on An Excel error led Austria's SPÖ to announce the wrong candidate as the winner
Even with plain text CSV files one can detect that a given cell was changed, have md5 checksums of files, etc. In a day to day use where users often share their edits using emails just figuring out which is the correct and most recent copy is a fools errand.

In some settings the sheets are just a giant white boards: different parts of the same sheet can contain partially replicated and not cross-checked data. Samples in columns, so each row is a different type. "Free flow of thought" or annotations of say numerical cell with text in the same cell.

All this steams from the fact that users do not see the data as something to be queried and compared but as a board one can just look at.

elmolino89··on An Excel error led Austria's SPÖ to announce the wrong candidate as the winner
If one has to write a program to check if a user did not screw up cut and paste from one sheet/column/whatever it would be IMHO easier to just skip Excel, user and just write a script automating error prone C&P in Excel.

If some user wants to view it, then the script can create a valid Excel file.

elmolino89··on Exercise Has Many Health Benefits. Weight Loss Isn't One of Them
You got an extra boost from increasing thyroid hormone levels: See i.e.: https://karger.com/etj/article/7/6/279/129157/TRH-Neurons-an...
elmolino89··on Exercise Has Many Health Benefits. Weight Loss Isn't One of Them
I guess the subject in question was not put in a prison cell where 800 kcal was handled through a controlled opening?

The 130kg person burning just 800kcal/day is something highly suspicious. I am not sure if would be enough to lie in coma in a +30C room with a severe thyroid hormones deficiency to get to that level.

There are examples from hundreds of concentration/POW camps, periods of famine: if such extreme BMR lowering would be a thing, we would see it long time ago.

elmolino89··on Exercise Has Many Health Benefits. Weight Loss Isn't One of Them
While we do burn some extra calories rebuilding muscles after say long session of cardio your numbers (at least for running) for 70kg person are OK. There may be also an appetite suppressing effect of exhausting training sessions.

There is no way that not even overweight but a middle aged mostly, sedentary person can do 10 miles runs 6x a week. IMHO one should start with biking and walking which are way easier on the joins and start running after getting to the level when 50km rides are not causing you any problems. This assuming your BMI is not way above 25.

elmolino89··on Wearing an eye mask during sleep improves episodic learning and alertness
The subject did sleep at their homes, which means the noise level was not controlled, and it seams not even measured (no noise word in the article). The effect may be real, but I would like to see it corroborated in a more controlled environment.
elmolino89··on Argbash – Bash Argument Parsing Code Generator
Stringing up a pipe with a mix of awk, tr, sed, cut, sort, uniq etc. is certainly not more readable than a Python script with comments not invoking any of these but using Python's data structures, or even data frames/SQL.
elmolino89··on Do heat pumps work in cold climates?
Air heat pumps obviously are way less effective when the temps outside are negative. The ground heat pumps work better, but the installation costs are the barrier. The silly thing is that even on a sunny winter day an air pump uses a fairly cold air instead of some heated by the sun collector. From what I got such air & sun collector systems are complex to manufacture and maintain, so the more energy hungry air heat pumps dominate.
elmolino89··on Do heat pumps work in cold climates?
We have such a thing (some Mitsubishi model I will check later on if needed\) at home in Catalonia. Works OK, but the house despite being well isolated in general has a weak spot: dark external blinds. Without some external shading these warm up in no time, so the air between blinds and windows goes probably +45C or more. To combat this the heat pump in the air condition mode has to work way harder.

So a bit like with a Victorian house in winter where isolation is the key: one has to start with making sure that the house interior gets as little heat from sun in the summer as possible.

elmolino89··on Ask HN: I'm 40 and feel my mental ability declining. Programming seems harder.
I second that. Without taking notes/diagramming no mere mortal can handle nested 9 levels just in the brain. And somehow I doubt that the OP colleagues are doing just that either. The diff between people being stuck and progressing is often how much and how smart they writings things down.

My suggestion would be to try to tackle the low hanging fruits first: find the parts of the lowest complexity, try to understand these and get the simplest things done first. This will also restore your confidence.

elmolino89··on We're wasting money by only supporting gzip for raw DNA files
Compression of FASTQ files can be greatly improved by sorting/clustering the reads. I use clumpify from BBMap for that. The bad: clumpify does not support zstd at this point.
elmolino89··on We need more water than rain can provide: refilling rivers with desalination
In many areas planting new trees without a stable source of already scarce water may be pointless. So, yes, desalination on a large scale will sooner or later would be necessary. At the same time idea of "provide more water, skip the water use regulation" is quite mad. Growing alfa-alfa in CA to be shipped to Saudi Arabia is brain dead even if apparently commercially profitable. It should not be.
elmolino89··on Aluminum formate Al(HCOO)3: Earth-abundant, scalable, & material for CO2 capture
From German Wikipedia the albedo of water changes a lot with the angle. Oceans already are "dark" when hit by sunlight at +45degs angles. So it may not be that important.
elmolino89··on Aluminum formate Al(HCOO)3: Earth-abundant, scalable, & material for CO2 capture
270B trees @22kg/year, right?

You cut them down each year and replant, or assume that there are no trees locking up 22kg CO2/year for more than 1 year?

elmolino89··on Aluminum formate Al(HCOO)3: Earth-abundant, scalable, & material for CO2 capture
>But paper is pretty effectively recycled and trees aren't the only input - transport, water, and heat at the plant. Exactly. Moreover 1ha of Arundo produces 20-50t or more dry mass per year packed in 100mx100m area. Granted, not harvested at 0% humidity, less dense than paper. Collecting 50t of recycled paper from the containers in the city probably needs more energy for transport.
elmolino89··on Aluminum formate Al(HCOO)3: Earth-abundant, scalable, & material for CO2 capture
Not if you got the plastic using fossil fuels and extra energy. Which is the case for most of the non-degradable plastic. Add to it that we are nowhere close to replace various types of plastics with materials obtained from non-fossil fuels.

Obtained from cellulose https://en.wikipedia.org/wiki/Cellulose_acetate does decompose fairly quickly in the soil.

elmolino89··on Aluminum formate Al(HCOO)3: Earth-abundant, scalable, & material for CO2 capture
Yes, moors would help. I remember that during some floods in Europe one of the ideas floating around was to construct large polders which will be flooded as needed. No clue if catching up flood waters that would be enough to turn such polders into carbon sink.

There was some documentary about a German farmer keeping his grasslands in a moor-like state by rising the water table lowered by drainage. This did work well for him in the times of the draught but in general if I remember correctly it did lower his income.

elmolino89··on Aluminum formate Al(HCOO)3: Earth-abundant, scalable, & material for CO2 capture
>For starters, humanity emits more CO2 than the combined mass of entire biosphere (every living organism, not just trees), every 10 years

""" Plants and soils together currently absorb an estimated 30 percent of the CO2 emitted by human activities each year. """

https://news.stanford.edu/press/view/38728

Do you have source for your 10 years?

elmolino89··on Aluminum formate Al(HCOO)3: Earth-abundant, scalable, & material for CO2 capture
Yes and no. Meaning: depositing fertilizer close to the shore in shallow waters causes toxic algal blooms and suffocates everything. One needs deep, cold waters so the sinking organic matter is not decomposing rapidly.

And yes, if the winds blow the Saharan dust over large oceanic surfaces algae do grow in otherwise "desert" waters and some of the generated biomass does sink deep.

elmolino89··on Aluminum formate Al(HCOO)3: Earth-abundant, scalable, & material for CO2 capture
I also think that compared to some pie or at times mirror in the sky terra engineering projects "plants for the rescue" look more realistic at this point.

>The biggest problem is the scale of it - to make a significant dent in global co2 you need to be pyrolysing a lot of wood

The giant scale and use of land is one thing. But increasingly in many places the limiting factor be it for agriculture or forest or biomass plantations is water. While forests and vegetation have more than just a sponge effect, we should keep in mind that a scorched dry forest or a field burns easily. One may look at it as +/- net zero (what burns was absorbed by the plants we planted), it may be safer to go for high biomass yielding plants we can cut preferably more than 1x a year. In the case of fire only the carbon absorbed in the last say half a year would be released. And nasty plants such as Arundo can re-sprout after the fire. Or in a case of a prolonged drought given the right equipment one can just cut it down and get rid of the danger.

*Edit*: cut one "While" too many

elmolino89··on Aluminum formate Al(HCOO)3: Earth-abundant, scalable, & material for CO2 capture
>Does this new scrubber material make it feasible to reduce the CO2 from the atmosphere, or will it be more economical to deal with the symptoms of high CO2 levels?

The low tech solution for scrubbing C02 from the atmosphere is to get more plants absorbing it and prevent the plant decomposition back to CO2 or even worse to methane. Some of the agricultural waste may be just buried in the fields helping to build the organic matter in the soil, but it would be hard to avoid any CH4 emissions doing it. Just check any landfill with buried organic matter and some sipping in water. One can try to use pyrolysis and get charcoal. This would not be 100% emissions free (CO, CO2, CH4 etc.).

The main problem with the above is that collecting wood/straw/etc. transporting and then processing it also requires energy, at least on land. Converting the millions of agro machines to run on batteries or hydrogen would not be carbon neutral either.

Supposedly if we fertilize patches of the oceans the dead algae will nicely sink to the bottom not to be release CO2 for centuries.

elmolino89··on Why most published research findings are false (2005)
Sure. Some genes were retired because there was not enough support. Earlier Ensembl versions used older genome assembly. Which means: some genes "jumped" from one chromosome to another, or got properly stitched residing before partially on floating contigs.

Just to be clear: I am not saying that anything done in 2022 using hg19 is 100% wrong. Just that it is a bit like using a stretched shoelace 50cm (+/- 5cm) to measure your corridor when you have a decent tape measure in your pocket.

There are microarrays used in Big Science projects with ~1/3 of probes not matching human transcripts from latest ENSEMBL. Since most of them map to the current genome who knows what is the meaning of the signal from such probes. Unannotated exons? Retained introns? But some probes do not even map to the genome => some silly splicing error(?) packed in a plasmid in the 90ies?

elmolino89··on Phantom forests: Ambitious tree planting projects are failing
The thing is we do not live in ordinary times. Last summer temperatures made many established trees in the cities close to getting killed. Anything with less developed root system and without regular, generous watering probably fared worse. Farmers hardly abandon their crops or are clueless about agriculture, but in many regions this growing season was a disaster.

From large scale, low initial investment and high yield tree planting: pick one. Any even the most amateur gardener can figure out that growing things successfully requires checking many boxes.

Last but not least: in climatic emergency maybe instead of as we can see a pipe dream of low cost, no maintenance, native species only eco-forest we may start to think about planting almost anything that grows successfully and does not burn too easily.

elmolino89··on Why most published research findings are false (2005)
While using the software packages released last week may not be the greatest idea, I am leery of any research paper using 10 years old genome assembly, Ensembl annotation release in the 40ish (we are at #105 ) or clearly outdated program versions with X updates in the last 5 years. Also if Fedex/UPS were tracking packages with bordering on cavalier attitude observed in some labs, often we would be getting bags of guano ordered by some horticulturalist instead of a book of our choice. Or even an empty bag, since QC may be an afterthought and hard wet lab work may still produce unusable crappy data.

On the other hand the brand new technologies are rather expensive, good quality human tissue samples hard to get so scrapping the bottom of the barrel trying to justify grant $$$ is unavoidable.

elmolino89··on pigz: A parallel implementation of gzip for multi-core machines
bgzf has a clear advantage when it comes to sorted BED/VCF/GTF formats, especially if one does index these. But frankly I have no idea if it may improve IO times for the fastq files when read by say bwa or another mapper. Do you have any experience with that? I have seen some mapping times improvement using fastqs with clustered reads (by clumpify).
elmolino89··on pigz: A parallel implementation of gzip for multi-core machines
Apart from tuning up the compression you may gain a lot by clustering reads in FASTQ files using clumpify from BBMap:

https://jgi.doe.gov/data-and-tools/software-tools/bbtools/bb...

You may also filter out duplicates and depending on what you do correct base errors or get rid of low complexity reads.

elmolino89··on Advanced Scientific Data Format
"bonus" points:

* text/TSV data files made public as a requirement from the published articles may have spaces or dots in column names, missing line ends, non-unique row IDs, etc.

* there is no limit as how many records can be stored in a single file. Latest dbSNP has more than 1000 millions of rows.

* bunch of formats (GFF, GTF, VCF) has several TSV delimited "proper" columns (as: 1 value) and then special column where optional fields are piled in a different format, with another separator, field names etc. Real fun to parse...

elmolino89··on Walmart, CVS face trial for homeopathic products next to real meds
I think in many countries one can not open a pharmacy without a proper paperwork/personnel with university degrees etc. IMHO it would be beneficial if every pharmacy store was visibly divided into "pharmacy proper" and "shampoo/water/wrinkles-be-gone cosmetics". Homeopathy should be expelled to tarot and crystal balls stores.
← PreviousPage 3 of 6Next →