felarof

Thank you! Yeah all user data is just stored locally on device.

Oh cool, will look into basic.tech to understand more.

Thanks for asking - not a stupid question at all! I should have probably explained it at the top of my post.

By "agentic browser" we basically mean a browser with AI agents that can do web navigation tasks for you. So instead of you manually clicking around to reorder something on Amazon or fill out forms, the AI agent can actually navigate the site and do those tasks.

Sounds good! will look into getting linux build.

Yes MacOS for now, but looking into getting Linux binary next.

> what's the reason for no Linux/Windows?

Sorry, just lack of time. Also we use Sparkle for distributing updates, which is MacOS only.

> Also what's the business model?

We are considering an enterprise version of the browser for teams.

> 100% runs locally

Thank you! We have ollama integration already, you can run models locally and use that for AI chat.

Thanks for the feedback. Honestly—we just reused the icon we had gotten professionally designed for the last idea we were working on (https://felafax.ai/).

But not gonna lie, as a tiny startup, we don’t have marketing budget of Perplexity or Dia, so we picked a name and icon that at least hinted at “browser” right away. Definitely not trying to mislead anyone -- just needed something recognizable out of the gate.

No, not today.

But wonder if it matter if it the agent is mostly using it for "human" use cases and not scrapping?

16GB M1 Pro is good enough to run our browser! You should give it a try!

Download form https://www.nxtscape.ai/ or our github page.

Haha, used gpt4o to generate it. What change do you want to see in that fox appearance? Any change should be a prompt away :)

Today, we connect to chrome using CDP and use Puppeteer to send clicks and other operations. Also, using browser use DOM tree highlighting, which works great.

To get the page content we parse accessibility tree.

Thanks for the feedback.

I get the general sentiment. But cursor for sure has improved productivity by a huge multiplicative factor, especially for simpler stuff (like building chrome extension).

Yes, we are building on top of chromium. haha noway two of us can build a new browser and feel it's not needed too.

There is a big red button to always stop the agent.

Haha, was easier to build and we were the first users :)

have linux next on our radar. What build do you want?

Yeah accessibility is one such usecase, but in future we have few other ideaswhere having a fork makes it lot easier. Few ideas:

- Ship a small LLM along with browser - MCP store built in

> The big question is whether demand for CUDA can be supplanted with application-specific accelerators.

At least for AI workloads, Google's XLA compiler and the JAX ML framework have reduced the need for something like CUDA.

There are two main ways to train ML models today:

1) Kernel-heavy approach: This is where frameworks like PyTorch are used, and developers write custom kernels (using Triton or CUDA) to speed up certain ops.

2) Compiler-heavy approach: This uses tools like XLA, which apply techniques like op fusion and compiler optimizations to automatically generate fast, low-level code.

NVIDIA's CUDA is a major strength in the first approach. But if the second approach gains more traction, NVIDIA’s advantage might not be as important.

And I think the second approach has a strong chance of succeeding, given that two massive companies—Google (TPUs) and Amazon (Trainium)—are heavily investing in it.

(PS: I'm also bit biased towards approach 2), we build llama3 fine-tuning on TPU https://github.com/felafax/felafax)

> Which means barrier to entry is very very high.

+1 on this. The tooling to use TPUs still needs more work. But we are betting on building this tooling and unlocking these ASIC chips (https://github.com/felafax/felafax).

dumb question: wdym by sparse work? Is it embedding lookups?

(TPUs have had BarnaCore for efficient embedding lookups since TPU v3)

+1, almost all (if not all) Google training runs on TPU. They don't use NVIDIA GPUs at all.

> CUDA, just the subset of tensor operations that are used for training and AI inference. If demand for training and inference wanes

Interesting take, but why would demand for training and inference wade? This seems like a very contrarian take.

TPUs have been used for training since a long time.

(PS: we are startup trying to make TPUs more accessible, if you wanna fine-tune Llama3 on TPU check out https://github.com/felafax/felafax)

+1, we still have a lot of performance we can extract! JIT-compiled train steps, more optimized data loading and sharding, gradient accumulation, and activation checkpointing. We will continue building and will do another blog soon after implementing all the improvements!

Tinygrad is great, but still in early stages I believe.

JAX has matured a lot over last 6 years and XLA has been around for lot longer. We believe we can extract good perf from AMD with JAX + XLA kernels.

We have a few technical issues that we still need to address:

1) This entire fine-tuning run was done in JAX eager mode. I kept running out of memory (OOM) when trying to `jax.jit` the entire training step. Even gradual `jax.jit` didn't work.

2) The current version doesn't have gradient accumulation, and with a batch size of just 16, that’s not ideal. I'm working on implementing gradient accumulation next.

3) We still haven't found a good way to load large sequence-length data (like 32k sequence length). Currently, before sharding the training batch across GPUs, it ends up loading the entire batch onto a single GPU’s VRAM and causes OOM issues.

Oops, my calculation was wrong. Let me add an edit to the blog, thanks for pointing it out!

My train step was taking 30s.

And I was using a batch size of 16 and seq length of 64, making the training speed as (16*64/30) tokens per sec == 35 tokens per second (for fine-tuning in JAX eager mode).

(I haven't done comparison with 8XH100)

Good question. We used a new AI+math-based testing tool (benchify.com) to run comparison tests, but we are working on building more robust infrastructure for this. Translating models from PyTorch to JAX is core to our strategy.

That said, this path is not uncommon (translating from one framework to another). HuggingFace translates Google's Gemma family models from JAX to PyTorch, and a ton of people use it.

Thanks for the feedback! I appreciate you pointing that out. My understanding was based on the PyTorch documentation for scaled_dot_product_attention (https://pytorch.org/docs/stable/generated/torch.nn.functiona...). - "The function may call optimized kernels for improved performance when using the CUDA backend. For all other backends, the PyTorch implementation will be used."

And was trying to make a broader point about the lack of transparency (in performance, lower-level impl) in PyTorch when running on NVIDIA vs. non-NVIDIA hardware.

Ahh, interesting, will take a look!

Curious what are the steps to run PyTorch on AMD (does it work out-of-box with PyTorch+rocm docker image)? Does torch.compile work smoothly?

Good question. No good metric give performance depends on software stack (JAX vs PyTorch) + optimizations.

But my take performance per dollar of TPU > AMD > NVIDIA.

Haha, TPUs are not slow :) All of Google's training (including Gemini models) is done on TPUs.

There are good 1p [a] and 3p [b] benchmarks comparing TPUs vs NVIDIA GPUs.

[a] - https://github.com/GoogleCloudPlatform/vertex-ai-samples/blo...

[b] - https://arxiv.org/pdf/2309.07181