HNHacker News
TopNewBestAskShowJobs

joshlk

446 karma · joined May 1, 2017

Views are my own.
submissionscomments
joshlk··on Dependency Confusion: How I Hacked Into Apple, Microsoft and Other Companies
Uploading dummy packages to PyPi isn't the solution. It just pollutes PyPi and a nuisance to others.

You have always been able to specify the `index-url` when installing packages using pip. This can also be added to `requirements.txt` files as well.

joshlk··on Command Path Security in Go
PATH is one of those puzzling concepts that shell beginners grapple with.

It’s a miracle that Windows and Unix converged a similar implantation.

joshlk··on Ballista: Distributed compute platform implemented in Rust using Apache Arrow
I think those are ok trade offs for a lot of projects but not for high performance and low latency applications such as databases. It seems to me that those project have shot themselves in the foot before even starting.
joshlk··on Ballista: Distributed compute platform implemented in Rust using Apache Arrow
I still don't understand why lots of new databases and data applications (Neo4J, Cassandra, Hadoop ...) are written in Java when it is well known that Java is not well suited to these types to tasks: GC freezes, bad memory model for large data, not fully compiled as so much slower when compared to C++, C or Rust. Why choose Java for these applications?
joshlk··on GPT-Neo – Building a GPT-3-sized model, open source and free
How does the outfit intend to fund the project? OpenAI spends millions on computing resources to train the models.
joshlk··on Tracing and visualizing the Python GIL with perf and VizTracer
Amazing article. Please keep them coming.
joshlk··on BeagleV – An affordable RISC-V computer designed to run Linux
What features makes it “designed for Linux”?
joshlk··on Thonny: A hassle-free Python micro-IDE
Looks similar to Spyder: https://www.spyder-ide.org/
joshlk··on On Repl-Driven Programming
You can also add breakpoints which automatically occurs when exceptions are raised. Which is similar to the breakloop functionality mentioned in the article.
joshlk··on Design flaws of password managers
Or if you have to change the password every X months
joshlk··on The impact of Apple Silicon Macs on Broadway
Unfortunately it's unlikely Linux will ever run on the new M1 Macs: "Linus Torvalds doubts Linux will get ported to Apple M1 hardware" https://news.ycombinator.com/item?id=25238175
joshlk··on SymPy 1.7
Out of interest, does anyone use SymPy outside the scientific/ academic community?
joshlk··on Ask HN: Top Coursera Courses?
Has anyone tried either?: - "Fundamentals of Parallelism on Intel Architecture" https://www.coursera.org/learn/parallelism-ia - "Introduction to High-Performance and Parallel Computing" https://www.coursera.org/learn/introduction-high-performance...
joshlk··on Ruffle – A Flash Player emulator written in Rust
How is this achieved? Is the flash API/architecture open knowledge?
joshlk··on Ask HN: Top Coursera Courses?
What does that mean to think in models?
joshlk··on Ask HN: Top Coursera Courses?
What’s good about it?
joshlk··on Ask HN: Top Coursera Courses?
Nand to Tetris: how a computer is built from the ground up. Starting from nand logic gates all the way to OO programming

https://www.coursera.org/learn/build-a-computer

joshlk··on Plugin hybrids in new emissions scandal: test show higher pollution than claimed
> FYI, here in tke UK, almost all Uber cars are older diesel saloons / estate cars, and mostly just a normal taxi with an extra "Uber" sticker.

That’s not true in London. Most Uber cars in London are Toyota Prius hybrid cars. I would guess that’s true for >80% of London Ubers

joshlk··on Plugin hybrids in new emissions scandal: test show higher pollution than claimed
Is this a reputable news source (genuine question)? Some of the lines are hyperbole: "Plug-in hybrids are fake electric cars, built for lab tests"
joshlk··on Ask HN: What is the best money you have spent on professional development?
Good idea. What do you mean exactly by improv lessons?
joshlk··on Run Python Applications Efficiently with malloc_trim
DataClassFrames also takes the approach of storing lists of dataclasses as a “dataclass of arrays” - know as data orientated design: https://gitHub.com/joshlk/dataclassframe

(Full disclosure I’m the author of the project)

joshlk··on Show HN: Dead easy Python for executing many HTTP requests asynchronously
I do actually - just dug it out: https://imgur.com/a/0uIQW6f This was an experiment I did against a very slow API. It's 5 years old and so I was using Python 2.7. The graph shows number of completed requests per second against number of threads. As you can see at first the throughput linearly increases with threads but then plateaus at around 12 threads.

I haven't done the same experiment with async-io but it would be quite easy using this package `many-requests`

joshlk··on Show HN: Pandas style guide for production code
A collections of recommendations that I have built up during my 5+ year experience using Pandas in a variety of commercial projects. I would like to ask the wider community for feedback or any other recommendations they would like to add. Feel free to start an issue thread or do a PR request.
joshlk··on Show HN: Dead easy Python for executing many HTTP requests asynchronously
That would be a good idea, but it's non-trivial in Python as multi-processing async support is patchy. If you have to choose multi-processing vs async for HTTP requests, from experience, async is much faster (as your not CPU bound but IO bound) and easier to use.
joshlk··on Show HN: Dead easy Python for executing many HTTP requests asynchronously
Its not simple to just "swap out" the requests library with an async one. This is why I wrote this package :) It deals with a lot of the headaches one would run into by trying just that.
joshlk··on Show HN: Dead easy Python for executing many HTTP requests asynchronously
All running in a single thread - but the work is executed asynchronously. This is preferred for I/O heavy throughput such as many HTTP requests.
joshlk··on Show HN: Dead easy Python for executing many HTTP requests asynchronously
This wont work for a large number of requests as the throughput plateaus quit rapidly with a large number of threads due to the GIL and blocking I/O. This is the main reason async was introduced into Python.

Also `many-requests` has further logic to deal with retries and bad responses.

joshlk··on Show HN: Dead easy Python for executing many HTTP requests asynchronously
Needed when executing 1 million API requests for example. Use the full force of your machine by utilising async IO via the excellent library Trio.

It has been tested in the wild with more than 10 million requests. Automatically handles errors and executes retries.

Also provides helper functions for executing embarrassingly parallel async coroutines.

joshlk··on Show HN: A Python container for dataclasses with multi-indexing and vector opps
This wasn't implemented as I think it would be very hard to do/impossible in Python.
joshlk··on Show HN: A Python container for dataclasses with multi-indexing and vector opps
Yes - you can use nested dataclasses. But you will loose some of the nice features
← PreviousPage 3 of 4Next →