You can actually download the weights without an account:

    git clone https://huggingface.co/bigscience/bloom

You will need Git LFS and ~330GB of free space though.

It is possible to run inference locally, even without very much RAM/VRAM (though it will be very slow). Get the most recent versions of "transformers" and "accelerate" from git, and then:

    import torch
    from transformers import AutoTokenizer, AutoModelForCausalLM
    
    tokenizer = AutoTokenizer.from_pretrained('./bloom')
    inputs = tokenizer("Hacker News is", return_tensors="pt")
    model = AutoModelForCausalLM.from_pretrained('./bloom', device_map="auto", offload_folder='offload', torch_dtype=torch.bfloat16)
    
    output = model.generate(inputs["input_ids"].to(0), min_length=30, max_length=30, do_sample=True)
    print(tokenizer.decode(output[0].tolist()))

After the model is loaded you can also look at where it put each part of the model (GPU, CPU RAM, or on disk):

    print(model.hf_device_map)

Here's a screenshot of it in action on my machine, which has 256GB RAM and 2xA6000 GPUs:

https://twitter.com/moyix/status/1546961979566325760