lkirk

I think this is a bit optimistic compared to my view (wrt portability). There's a large stack of software that is involved in training and probably less so in inference. I'm not saying it's impossible but there are definitely different levels of reproducibility and the academic incentive structure doesn't really prioritize reproducibility in my experience. I'm sure it varies quite a bit, I'd be curious to know how those in this problem space are thinking about reproducibility and at what level.

For me, in my use of Python as a data analysis language, it's not python's speed that is an annoyance or pain point, it's the concurrency story. Julia's built in concurrency primatives are much more ergonomic in my opinion.

This thought led me to a strange conversation with chatgpt. I just base64 encoded "Hello chatgpt how are you" as "SGVsbG8gY2hhdGdwdCBob3cgYXJlIHlvdQ==" and sent it as the first prompt. Notice how there is no question mark in my phrase. Now here's my conversation with chatgpt:

User SGVsbG8gY2hhdGdwdCBob3cgYXJlIHlvdQ==

ChatGPT This appears to be a Base64 encoded string that translates to "Hello chattpt how are you?"

User not quite... almost but not quite

ChatGPT I apologize for the mistake. The encoded string actually translates to "Hello chatgpt how you are".

User also not quite, you were closer the first time

ChatGPT Thank you for letting me know. The correct decoded message for the given Base64 encoded string is "Hello chatgpt how are you?" I apologize for any confusion my earlier responses may have caused.

User can you now re-encode it into base64

ChatGPT Certainly! The message "Hello chatgpt how are you?" encoded in Base64 is:

SGVsbG8gY2hhdGdwdCBob3cgYXJlIHlvdT8=

I hope this helps!

I find it particularly odd that it was misspelling things in its decoding and adding punctuation where there was none. I suppose a LLM would be trained to do something like that, but the "chattpt" really threw me off. Also, it's base64 encoded message back to me decodes as "Hello chatgpt how are you?". Maybe not that impressive of a demonstration, but definitely not was I was expecting.

Yes, I would agree that technical and non-technical usability is one appealing spreadsheet use case. I think that another thing I find myself using spreadsheets for is simple data entry. By using a google sheet to enter data, you've already got yourself an authenticated, multi-user web form for easily entering data. It even integrates with google forms, which is a very handy tool for collecting data. It's certainly a lot easier than building it all yourself and hosting it.

Once the data is entered, there are much better tools for data visualization and aggregation (in my opinion). It's nice to have the "frontend" taken care of so that your time is freed up to analyze and extract meaning from the data. I'm speaking from the perspective of someone who does a lot of rapid prototyping and data analysis. There are likely use cases where this won't scale.

Is this something that the internet archive would preserve?

Can you elaborate a little here? I made the switch from classic notebooks to jupyterlab recently and find the bin experience very similar (there's a few subtle differences that I can't remember offhand, but I don't recall having any problems with cell navigation)

In my experience, there's a balance to be stricken. I really like notebooks for documenting the algorithm development process. I used to do a ton of repl driven development and Jupyter is a repl that allows you to persist commands across sessions. It saves a ton of time that I used to spend scrolling through the ipython history after closing and restarting the session. Jupyter also allows you to manage different kernels in the same environment, so it makes tasks like testing code between py2 and py3 trivial. My final point here is that there's also excellent cython integration, so you can do a lot of prototyping of cython code without having to mess with configuration or multiple files. I will agree that there are tasks better suited to IDEs, but jupyter is not just a plotting frontend, it can be used very effectively in algorithm development and the communication/documentation of the development thought process.

Typically, I have a git repo with the final code products, some of the more complex code gets written in notebooks, then transferred to git and thoroughly tested. I've been dreaming of this debugging experience in jupyter because that's still not a task that's suitable for notebooks, but I am hoping that it will come for vanilla python kernels before I can hope to adopt it.

A sequencing run (at least in the context of Illumina's technology) requires a few very expensive consumable reagents: First being the flowcell (microscope slide that the dna sticks to while being read by a laser), the reagents (containing enzymes with fluorophores and other reagents for amplifying and manipulating DNA), and the actual power consumed by such machinery. This does not factor in prep/lab costs (which can be kept at a minimum with automation, but that also is a high startup cost endeavor). Each sequencing run can take ~1-3 days depending on the format.

Edit: this video may be able to explain a little better how this process works: https://www.youtube.com/watch?v=fCd6B5HRaZ8

Nanopore sequencers may require less prep and may be cheaper, but their sequencing error rate is astronomical, so you'd end up doing a lot more sequencing of the same material before you reached a consensus sequence.

Indeed, one could only imagine the cost of sequencing the dna to retrieve the data (not to mention the current lack of random access). Illumina's highest capacity sequencer will do 6Tb (terabases). The machine costs about half a million dollars and each run is tens of thousands of dollars, not to mention the lab costs of preparing/storing the dna. Additionally, the depth at which one would have to sequence to get _all_ of the data back reliably would be >1 meaning that every base would have to be sequence more than once (to avoid sequencing errors).

I think the biggest thing, for me is Cython. I've not seen anything quite like it in other languages. It allows you to compile python code to c, with gradual typing. It also allows you to write c code inline w/ your python or interface with other C/C++ libraries. https://cython.org/ Other languages will be pressed to beat its utility (esp in the scientific computing world)

I thought that it was implemented in Arc: http://arclanguage.org/

A big use case for python is glue for high performance c/c++ code. How is racket at this? Also, is there a cython alternative for racket? I would also argue that, though racket may have more batteries included, there are definitely not as many externally developed libraries. Especially important are numpy/scipy/pandas/pysam, the list could go on... That's not to say that I'm against racket, I really like the language and its level of design and documentation.

It would be nice if this language was extensible to running on various compute cluster managers. From what I can tell, these workflows only run on one machine. I like the bioinformatics tool examples though... you can tell who their target market is ;P

I've heard of it, not sure how mature it is these days. Maybe I'll give it another look

I'm also a heavy user of an X-only window manager (xmonad). Learning about this makes me wonder if I should start learning how to work with another window manager so I'm not caught off guard when something stops working.

Sorry, but that's not a vcf. It's a tsv of genotypes. Here's the spec for VCFs: https://samtools.github.io/hts-specs/VCFv4.3.pdf

Very cool, thanks. I will probably be using this in the future

I have used this in the past. There are only a few places where this bites you, for instance wifi authentication portals. Some of these wifi access points change your resolv.conf so that you can load their internal page for terms and conditions.

The title of the site is "Matrix - ITA Software by Google". Also, on the site, they link themselves:

  Want to explore flights and get fast results?
  Try Google Flights

Almost seems like a deprecation notice...

It really depends on how light it is outside. I like light backgrounds when it's light outside, but dark ones at night.

I think people also forget that there's a decent amount of fortran in numpy. I'm pretty sure they are the originators of f2py as well

A pid file is the best IMO

Yes, these are the same

Noooooooooooo :(

Just an FYI: the newest version of ipython allows you to tab complete keys

Anything from Jupiter broadcasting, but specifically the Linux action show and bsd now

http://www.jupiterbroadcasting.com