HNHacker News
TopNewBestAskShowJobs

neuralll

1 karma · joined April 21, 2021

submissionscomments
neuralll··on Show HN: Llama.cpp fork with 2-4x multiGPU speed for MoE models bigger than VRAM
I'm definitely trying. I've already posted the work in the llama.cpp PR thread, and a few collaborators have started testing it. Hopefully we can turn it into a proper PR and get it merged upstream.

As for the job-seeking part: I won't deny that I'm looking for opportunities. I'm currently an unemployed AI developer and could certainly use a job offer. But that wasn't the main reason for posting.

A few years ago I sold my house(which made me homeless) and moved to Australia after being told that completing an AI Master's degree there would provide a pathway to a post-graduate visa and eventually citizenship. After spending roughly $40k on tuition and living costs, the eligibility rules changed and the maximum age for that visa category was reduced from 48 to 35, leaving many international students without the pathway they had planned for. So yes, I'm job hunting, but this post wasn't intended as a recruitment ad.

The real motivation was technical excitement. Seeing models like GLM-5.3, K2, MiMo-2.6, DeepSeek and similar large MoE models approaching frontier-level capabilities while running locally at nearly 28 t/s on just 2×3090s plus DDR4 RAM felt genuinely liberating after years of dealing with cloud costs and usage limits.

I suspect many people with two or more GPUs will feel the same way. If we can make multi-GPU systems contribute compute instead of mostly acting as extra VRAM, a lot of users suddenly gain access to much faster local inference on models that previously felt impractical. That's really why I wanted to share it.

neuralll··on Show HN: Llama.cpp fork with 2-4x multiGPU speed for MoE models bigger than VRAM
I built a llama.cpp fork that fixes a real problem: with stock llama.cpp, extra GPUs mostly just add memory, not speed, for MoE models too big for VRAM only one GPU computes at a time. Mine adds a live expert cache so multiple GPUs actually compute in parallel with the CPU. Measured on 2x RTX 3090: 12.3 to 27.7 t/s on GLM-5.3-Flash (117GB), 4.7 to 9.9 t/s on MiMo-2.6-Flash (132GB) — 2-2.3x, same output quality. Builds on an existing open PR (csantiago78's GPU expert-cache PR #27861) — credited prominently in the README, along with GLM-5.3-Flash support from timkhronos's PRs. Happy to answer questions about the implementation. Looking for Job abroad too as war is looming in EU
neuralll··on According IRS Startups Income
thx troy. but the 109$ fine is not imposed by regulations. its imposed by 3rd party e-filing agent that on IRS page claims that he is free e-filling agent for taxpayers and benefits from free advertising by being redirected to from IRS page itself. And infinite supply of scared IRS taxpayers due to hefty IRS fines. But in reality they are not free and ask 109$ for federal tax return from taxpayer having income less than 72k. Which according to IRS page they shouldn't.
neuralll··on According IRS Startups Income
yes but what about startups which are typically c corps with zero income or loss ? I was essentially forced to incorporate by advice on Indiegogo page for unsupported countries (slovakia) just to get access to crowdfunding. where GoFundMe or Indiegogo will support us business entity I create. Mine head is still spinning from the possible fines I am now threatened with if I miss some IRS tax filling deadline. 25000$ for not filling 5472 in time and up to 45000$ for not filling be13 in time. while i now have c corp . But Mine US bank account was denied by SVB with "slovakia unsuported" argument again. And all I wanted is to find funding for mine reusable covid test that I will always publish for free since goal is to save as many people as possible and not make money.It did passed to semifinals of XPrize but not finals https://www.protocols.io/view/covid-19-indirect-detection-th... but without funding I simply cant continue . I cant afford bulk PC or nylon filter and 120nm testing nanoparticles prices on my own anymore. All mine savings gone to all prototypes during XPrize months and I am now broke.https://ladislavnevery.medium.com/how-i-tried-to-save-a-worl... That's why I am so sensitive now to fees. And why I see being forced to pay 109$ just to e-file zero income loss report to IRS as immoral when IRS clearly states taxpayers with income less than 72k should not be charged. And filing internationally by snail mail there is no return receipt and risk of fine if mail is lost.
neuralll··on According IRS Startups Income
citation from FAQs - Free File Alliance “Free File preparation and e-filing assistance is available to taxpayers who have a 2020 Adjusted Gross Income (AGI) of $72,000 or less.” citation from IRS page above "As long as you qualify for the Free File federal return offer, you must not be charged for preparation and e-filing of a federal tax return" Startups and small businesses are taxpayers. These are hard times. Many LLC small businesses are defaulting or going out of business due to covid as we speak. Many startups didn’t receive EIN and had zero revenue or loss (450$ Delaware franchising tax?800$ CA Tax).

109$ via cheapest TaxAct to send zero revenue or loss to IRS or face 25000$ fine for not filling 5472 is absurd

IRS search shown 6 agents.

Do they provide free e-filling for all taxpayers with <$72k income? = startups or small business ?

freetaxact.com no freefile.intuit.com no freetaxusa.com/freefile2020 no 1040now.net/freefile.htm no taxslayer.com/americanpledge no freetaxes.free1040taxreturn.com no

I filled IRS feedback to form 1120 got no response Apr 3, 2021, 5:54 PM got just Robotic Non-Response 1 from helpdesk@freefilealliance.org: Apr 3, 2021, 6:30 PM Robotic Non-Response 1 from phishing@irs.gov

After lengthy email exchange with TaxAct I got sentences "Also, the IRS FREE FILE is for individual taxpayers, not businesses." or "if you use the IRS Free File | Internal Revenue Service product look up tool, you will see there is no category for business taxes" to which I replied If IRS or Free Filing Alliance actually meant that only individual taxpayers are eligible for free federal tax filing. They would simply clearly said so by using word “individual taxpayer” everywhere they used word “taxpayer” in sentences defining eligibility for free filing."

The best would be calling IRS and Free Filing Alliance by phone due to responding to emails.But mine engliish. sux.

Will some brave soul rise to the occasion?