kernelsanderz

Stability AI became a $1 billion company with the help of a viral AI text-to-image generator and — per interviews with more than 30 people — some misleading claims from founder Emad Mostaque

Adding this to the huggingface peft library would be amazing. That's the main library that people using LoRA are currently using. https://github.com/huggingface/peft/issues/289

If you look at the dockerfile you can see that it’s copied from alice.py

I'm excited to see this get implementations for mobile platforms. a 30MiB model sounds like a good tradeoff to stream audio over low bandwidth, or even less reliable mobile connections.

Did some more research to see if anything had changed in this space. I found two interesting projects (haven't used them myself yet though):

One in C# (with support for auth)

https://github.com/alanedwardes/Estranged.Lfs

One in Rust (but no Auth, have to run reverse proxy)

https://github.com/jasonwhite/rudolfs

Both seem interesting. Anyone use these?

Last I checked it wasn't easy to use something like optuna to do hyperparameter tuning with hydra/DVC.

Ideally I'd like the tool I use for data versioning (DVC/git-lfs/gif-annex) to be orthogonal to that which I use for hyperparameter sweeping (DVD/optuna/SageMaker experiments), and orthogonal to that which I use for configuration management (DVC/Hydra/Plain YAML), to that what I use for experimental DAG management (DVC/Makefile)

Optuna is becoming very popular in the data-science/deep learning ecosystem at the moment. It would be great to see more composable tools, rather than having to opt all-in into a given ecosystem.

Love the work that DVC is doing though to tackle these difficult problems though!

I'd love to see a well-supported git-lfs compatible client/proxy (so you could more easily move backends) that could run on top of S3/object storage. Yes, and written in a modern language like golang/rust for performance / parallelism. There's some node.js and various other git-lfs proxies out there, but not well enough maintained that I could count on them being around and working in another 5 years. git-annex at least has been around for a while, even though it has its issues.

Huggingface uses git-lfs for large datasets with good success. git-lfs on GitHub gets very pricey at higher volumes of data. Would love the affordability of object storage, just with a better git blob storage interface, that will be around in the future.

Most of these systems do their own hash calculations and are not interchangeable with each other. I feel like git-lfs has the momentum at the momentum in data-science at the moment, but needs some better options for people who want a low cost storage option that they can control.

Huggingface is great, but it's one more service to onboard if you're in an enterprise. And data privacy/retention/governance means that many people would liek their data to reside on their own infrastructure.

If AWS were to give us a low cost git-lfs hosted service on top of S3 it would be very popular.

If anyone knows of some good alternatives, please let us know!

I do feel like git-lfs is a good solution. Once you have 10s or 100s of GB of files (eg. a computer vision project), this gets pretty pricey.

Ideally I'd love to use git-lfs on top of S3, directly. I've looked into git-annex and various git-lfs proxies, but I'm not sure they're maintained well enough to be trusting it with long-term data storage.

Huggingface datasets are built on git-lfs and it works really well for them for storage of large datasets. Ideally I'd love for AWS to offer this as a hosted thin layer on top of S3, or for some well funded or supported community effort to do the same, and in a performant way.

If you know of any such solution, please let me know!

Here are two other approaches that I’ve stumbled on for doing similar things:

- https://www.cmcrossroads.com/article/rebuilding-when-files-c... - http://olipratt.co.uk/rebuilding-makefile-targets-only-when-...

This sounds like a great initiative. I can't find anything on the site about a privacy policy, how the email addresses you're collecting will be used. If you're creating this for the AI community, then how will this data be made available to others?

You kind of can - projects like deepspeed (https://www.deepspeed.ai/) enable running a model that is larger than in VRAM through various tricks like moving weights from regular system RAM into VRAM between layers. Can come with a performance hit though depending on the model, of course.

I’m wondering if you could compile make to wasm and include it as a cross platform dev dependency?

well, apparently in 2018 Google signed a secret deal with Mastercard sharing credit card payment information to do offline tracking of sales that originated from their ads. source: https://www.bloomberg.com/news/articles/2018-08-30/google-an...

and they did this, even if you told them not to track you.

As someone who has recently had to switch from Google Sheets to Office365 the experience has been very poor. Even things like keyboard shortcuts are very inconsistent with the web version of Excel. And if someone is editing the document while people are editing the web version watch out, random things can happen.

Example: How do you enter a cell to edit in Excel? On the desktop (I'm a mac user), it's ctrl-u (yep - ctrl). On the web I can't find any shortcut. I have to double click the cell like an animal.

Example: How do you select all the text inside a cell. On the mac it's cmd-a as you'd expect, on the web it's ctrl-a. Sigh.

The seamless collaboration of Google's apps is a joy, though as you've indicated, you give up a lot of features and power.

I clicked thinking it was some NLP project that ran on the GPU.

ncdu saves my bacon at least every few months. I do machine learning and am always running out of space!

I love the idea of Spyder. Particularly as a replacement/alternative to Jupyter notebook style workflows. The last time I tried to use Spyder to connect to a remote kernel, there was no way to interrupt a remote kernel. I understand that this was an inherent issue with the kernel APIs (Jupyter has it's own watchdog process that is responsible for sending signals to the kernels it monitors). Is this workflow better supported now? (ie. using Jupyer to connect to a remote kernel that has GPUs?, and having an easy way to interrupt / restart the remote kernel if it hangs?

I use a vim plugin called slimux to send code from vim into a REPL running in another tmux pane. It works really well.

Here's the fork that I'm currently using https://github.com/grusky/slimux

I use it with python and node.is and MySQL and psql

It's great because it can work with any REPL.

I used to use jupyter notebooks but almost exclusively use slimux now unless I need to some data visualisation or computer vision.

VS code's ipython REPL is quite good and has support for images and graphs.

I will say that the main thing that triggered a REPL driven workflow was dealing with large state. When you do machine learning and you have to deal with large models and datasets, loading them each time you save your code and triggering a unit test really gets tedious. You can definitely do it with caching and reducing the dataset size, but it only gets you so far IMO. There's nothing like being to inspect the state of your environment. Particularly when it takes a long time to create that state.

I this intended for inference of for training?

This is a great milestone for Ampere GPU owners, but to get the most out of Ampere you’ll need to wait for PyTorch 1.8 which should have CUDA 11.1 - which has better performance. You can compile from source to get that today. Also CUDNN 8.04 is not Ampere optimized either. We’ll have to wait until November to see CUDNN 8.05 or 8.1 which will have better Ampere performance for 3090/3080 GPUs.

You might also want to check out https://github.com/NodeOS/NodeOS

I was originally appalled at the software limiting. But according to Tim Dettmers who has a solid record of predicting and comparing NVIDIA cards for deep learning performance, it's not really a big deal.

You can read his analysis here: https://timdettmers.com/2020/09/07/which-gpu-for-deep-learni...

and his tweet about this here: https://twitter.com/Tim_Dettmers/status/1311354118514982912

Essentially from my understanding it's memory bandwidth which is the real critical path on performance in most cases. The previous generation of Turing cards had more compute than was necessary so they were an underutilized resource.

Also, this Puget benchmark is using an older version of the CUDA drivers. I believe performance is much better in CUDA 11.1.

This new benchmark which is running on the latest CUDA seems to confirm Tim's numbers: https://www.evolution.ai/post/benchmarking-deep-learning-wor...

This is very interesting work. My concern is that could this technology be used to test for depression, and might employers require their workforce to be tested, and discriminate on this basis? I guess measuring brainwaves is relatively involved. But that was my first thought after - Wow, that's impressive work!

Doing a quick search of TESS2 for trademarks look they already have the trademark registered for ZEIT (registered in 2017). So it was probably for other reasons. Which is a shame, because Zeit is very concise, easy to remember, and has a strong brand today - at least with their initial audience.

Sounds like they're trying to go for a more corporate customer with their new name.

Do these libraries use deep learning models? It's not entirely clear from the doco.

I run Linux (latest Kali Linux) on a similar MacBook Air and it work great for any of the light container tasks I've thrown at it (Docker, firecracker-vm, etc)

This.

It would be nice to fork the project and do something similar with CloudFront. Any static object data can also be fetched from CloudFront as JSON files, and periodically updated by cache invalidation or cache expiry dates (ie. cache for 5 mins).

Reminds me a kickstarter project I backed years ago called ScanBox. I still pull it out from time to time to scan documents when traveling, and have used it once or twice for a live document scanner, though the sides of the box make it difficult to get your hand into.

These days our team either uses the built-in whiteboard functionality in Zoom, or jump into a Google Drawing or Slides document for the live collaboration.

Sometimes I'll also use Zoom's AirPlay functionality to share my iPad screen and use an app like Notability to share. Though that's only one-way and not collabarative.

That's what I thought. Surely you could buy 10 professional document cameras for the value of that pasta! :-)

I think this is a great idea, and applaud the idea of bringing a high availability, high-load site template available to others.

The number of times I have thought in the past few weeks that if they had just used some static pages on S3 behind Cloudfront, or some kind of CDN, that much pain could have been averted.

Of course the first thing I did was to benchmark the test site to see how their edge network performs. For reference I'm based in Melbourne, Australia, and have a 100mbps download, 50mbps upload connections:

  $ ab -n 10000 -c 100 https://emergency-site.dev/
  This is ApacheBench, Version 2.3 <$Revision: 1826891 $>
  Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
  Licensed to The Apache Software Foundation, http://www.apache.org/
  
  Benchmarking emergency-site.dev (be patient)
  Completed 1000 requests
  Completed 2000 requests
  Completed 3000 requests
  Completed 4000 requests
  Completed 5000 requests
  Completed 6000 requests
  Completed 7000 requests
  Completed 8000 requests
  Completed 9000 requests
  Completed 10000 requests
  Finished 10000 requests
  
  
  Server Software:        Netlify
  Server Hostname:        emergency-site.dev
  Server Port:            443
  SSL/TLS Protocol:       TLSv1.2,ECDHE-RSA-AES128-GCM-SHA256,2048,128
  TLS Server Name:        emergency-site.dev
  
  Document Path:          /
  Document Length:        4836 bytes
  
  Concurrency Level:      100
  Time taken for tests:   106.534 seconds
  Complete requests:      10000
  Failed requests:        0
  Total transferred:      53220000 bytes
  HTML transferred:       48360000 bytes
  Requests per second:    93.87 [#/sec] (mean)
  Time per request:       1065.345 [ms] (mean)
  Time per request:       10.653 [ms] (mean, across all concurrent requests)
  Transfer rate:          487.85 [Kbytes/sec] received
  
  Connection Times (ms)
                min  mean[+/-sd] median   max
  Connect:      713  808  30.7    803    1828
  Processing:   230  236   4.8    236     443
  Waiting:      230  236   3.9    236     310
  Total:        956 1044  31.7   1039    2067
  
  Percentage of the requests served within a certain time (ms)
    50%   1039
    66%   1047
    75%   1053
    80%   1057
    90%   1070
    95%   1082
    98%   1107
    99%   1168
   100%   2067 (longest request)

I know there's much better ways of testing load/performance. It's just what I had on hand.