HNHacker News
TopNewBestAskShowJobs

maciejgryka

571 karma · joined April 23, 2009

http://maciej.gryka.net
submissionscomments
maciejgryka··on Building reliable systems out of unreliable agents
Ha, we tried that! Didn't make a noticeable difference in our benchmarks, even though I've heard the same sentiment in a bunch of places. I'm guessing whether this helps or not is task-dependent.
maciejgryka··on Building reliable systems out of unreliable agents
This is a bunch of lessons we learned as we built our AI-assisted QA. I've seen a bunch of people circle around similar processes, but didn't find a single source explaining it, so thought it might be worth writing down.

Super curious whether anyone has similar/conflicting/other experiences and happy to answer any questions.

maciejgryka··on ScreenAI: A visual LLM for UI and visually-situated language understanding
I haven't read through it yet, but there's FerretUI from Apple (mobile-specific, but I think a lot of learnings are generic) https://arxiv.org/abs/2404.05719
maciejgryka··on GPT-4 Turbo with Vision Generally Available
My experience is that it's pretty good at reading text and pretty bad at understanding layouts. So e.g. asking it to work with tables is asking for trouble.
maciejgryka··on GPT-4 Turbo with Vision Generally Available
Competition is great! But in this case, I don't know, adding JSON mode and function calling was a pretty obvious next steps for the vision model - I bet it'd have happened anyway.
maciejgryka··on GPT-4 Turbo with Vision Generally Available
He, cool to hear you're doing something like this too! We ended up close, but we also need good spatial relationships, which GPT4V isn't great at, so we're using other OCR system and adding the result to the context.
maciejgryka··on GPT-4 Turbo with Vision Generally Available
It's cool to see this update and JSON mode and function calling will both be useful with vision. I wonder, though, if there were any other specific changes to the models since the `preview` versions besides that?
maciejgryka··on Launch HN: Encord (YC W21) – Unit testing for computer vision models
Congrats on the launch!

I haven’t had a chance to try out Active yet, but having had a project with Erik and the team a while back, they’re a great team to work with :)

maciejgryka··on AMD's Next GPU Is a 3D-Integrated Superchip
Oxide and Friends just did an episode talking about this: https://oxide.computer/podcasts/oxide-and-friends/1643335
maciejgryka··on Benchmarking GPT-4 Turbo – A Cautionary Tale
I have similar conclusions so far. We have a custom data set (basically visual Q&A about web apps) and `gpt4` gets roughly 90% correct, while `gpt-4-1106-preview` only 86%. It's a little noisy (I didn't yet check out the new seeds functionality), but roughly consistent.

Since I created this dataset by hand, it can't really be memorized. I'm sure there's _similar_ data in the training set, but answering correctly still requires some reasoning-like capabilities.

maciejgryka··on Elixir and Rust is a good mix
Shameless plug & also a point of support for the article: I used that same stack to build https://regex.help/ (more details here https://maciej.gryka.net/building-regex-help)
maciejgryka··on RegExr: Learn, Build and Test Regex
If I understood what you mean, then yes, I built one https://regex.help/ (powered by https://github.com/pemistahl/grex doing the heavy lifting).
maciejgryka··on Launch HN: Rainforest QA (YC S12) – No-Code UI Test Automation
Oh thanks for letting us know! Where did you find this link? It should be https://help.rainforestqa.com/
maciejgryka··on Launch HN: Rainforest QA (YC S12) – No-Code UI Test Automation
For sure, that's weird - email support@rainforestqa.com and we'll sort you out.
maciejgryka··on Launch HN: Rainforest QA (YC S12) – No-Code UI Test Automation
Yes, we are dogfooding pretty hard! We release a bunch of times a day and every release goes through a suite of >130 Rainforest tests for regression. We also add tests for every new feature we release, so it keeps growing.

It was a cool thing to write tests for testing the test writing interface :) That's one "level deep" - out of curiosity I was playing with how deep it's possible to go and it turned out at 7 levels things get too small to really make sense (terminal within a terminal within a terminal etc.), but there's no inherent limit.

And we can do this because we work on the pixel level - even though the terminal is a canvas element and it'd be hard/impossible to work with it using the DOM, it doesn't really matter for KVM.

maciejgryka··on In search of the perfect URL validation regex (2010)
Yeah, grex (the library powering this) is really cool, but doesn’t generalize very well. I’m sure there are ways to improve it, but it’s not a trivial thing to do.
maciejgryka··on In search of the perfect URL validation regex (2010)
Using https://regex.help/, I got this beauty which passes all the ones, which should pass. Obviously some room for improvement ;) But it works!

  ^(?:http(?:(?:://(?:(?:(?:code\.google\.com/events/#&product=browser|\-\.~_!\$&'\(\)\*\+,;=:%40:80%2f::::::@ex\.com|foo\.(?:bar/\?q=Test%20URL\-encoded%20stuff|com/(?:\(something\)\?after=parens|unicode_\(\)_in_parens|b_(?:\(wiki\)(?:_blah)?#cite\-1|b(?:_\(wiki\)_\(again\)|/))))|uid(?::password@ex\.com(?::8080)?/|@ex\.com(?::8080)?/)|www\.ex\.com/wpstyle/\?p=364|223\.255\.255\.254|उदाहरण\.परीक्षा|1(?:42\.42\.1\.1/|337\.net)|مثال\.إختبار|df\.ws/123|a\.b\-c\.de|\.ws/䨹|⌘\.ws/|例子\.测试|j\.mp)|142\.42\.1\.1:8080/)|\.damowmow\.com/)|s://(?:www\.ex\.com/foo/\?bar=baz&inga=42&quux|foo_bar\.ex\.com/))|://(?:uid(?::password@ex\.com(?::8080)?|@ex\.com(?::8080)?)|foo\.com/b_b(?:_\(wiki\))?|⌘\.ws))|ftp://foo\.bar/baz)$
I had to replace some words with shorter ones to squeeze under 1000 char limit and there's no way to provide negative examples right now. Something to fix!
maciejgryka··on Saying Goodbye to Typo CI
Sounds like a smart decision and, while it sucks to have things not work out, I’m sure the author will do more cool stuff down the line.

For everyone interested in “small SaaS” (for the lack of the better term), I really recommend listening to the “Software Social” podcast, co-hosted by Michelle Hansen, mentioned in the article: https://mjwhansen.com/software-social-podcast/

maciejgryka··on Building Regex.help with LiveView and Rustler
This was a fun project to build and I thought some of y'all might find it helpful to use. Keen to get feedback either way!
maciejgryka··on Playing Codenames with Language Graphs and Word Embeddings
Fun to see what this paper did - and Codenames itself is great!

It's best in person, of course, but it also works really well online with a bunch of friends. There's a free, official, online version here https://codenames.game/

And also, shameless plug, I built a clone in Elixir for fun a couple weeks ago: https://maciej.gryka.net/building-secretwords

maciejgryka··on Show HN: Places to watch ducks if you're sick of everything
Delightful. I really like the trend of single-purpose, “artsy” websites, it feels like there’s been more of them recently.

...someone should make a site, which lists them all.

maciejgryka··on Mattermost’s QA Journey with RainforestQA
I lead the automation efforts at Rainforest and it's always interesting to see how our customers use our product to implement their QA.

This is a nice example of how Mattermost is using both Rainforest and Cypress in their release process. There are pros and cons of both and if anyone has question/ideas I'd be happy to discuss.

Plug: here's an intro to our automation product if anyone's interested https://www.rainforestqa.com/automated-testing

maciejgryka··on Using Python and Selenium for automated visual regression testing
Very interesting to see this! From the comments it seems that a bunch of people were burned by Selenium-style testing in the past. There's a pretty interesting paper/thesis [0] about this which talks abut what kind of breakages are most common in these scenarios.

Which is why (shameless plug alert) we at Rainforest [1] are working with crowdsourced testers - humans are still much, much better at visual diffing and judgement than machines are. Feel free to shoot me an email if you're interested in talking about it and figuring out how much to automate and how much to leave to humans.

[0] Why do Record/Replay Tests of Web Applications Break?, Mouna Hammoudi, Gregg Rothermel, Paolo Tonella

[1] https://www.rainforestqa.com/

maciejgryka··on Moving from Heroku to Google Kubernetes Engine
We are hiring :) https://rainforestqa.com/careers
maciejgryka··on Masters of Love: Lasting relationships come down to kindness, generosity (2014)
Right, I wouldn’t argue that it’s “only” anything! Like any human issue, there are many complex factors, including the fact that some matches are better than others. But I think our culture over-emphasizes finding the perfect match and totally ignores the fact that given any two people, their chances of being happy together can dramatically increase if they spend time and effort learning.

The idea that you need to find the perfect match, be totally yourself all the time and spend no effort making compromises is toxic. You’ll never have those expectations met.

maciejgryka··on Masters of Love: Lasting relationships come down to kindness, generosity (2014)
Check out the book “The Course of Love” by Allain de Botton. I don’t think the problem is that the parents were happy! It’s that they were apparently unable to teach the being-happy-in-a-relationship skill to their kids. And it is a skill: more natural for some than others, but possible for everyone to learn and improve on with time and effort.
maciejgryka··on Instagram announces IGTV
I've recently read a report about how media optimized for addictiveness impacts children's development [0] and found this sentence from the IGTV announcement particularly jarring: "Just like turning on the TV, IGTV starts playing as soon as you open the app".

[0] https://5rightsframework.com/static/5Rights-Disrupted-Childh...

maciejgryka··on Ask HN: Who is hiring? (February 2018)
RainforestQA (YC12) | Data Scientists, Software Engineers, Ops, other roles | San Francisco + the world | REMOTE | Full-Time We're hiring Data Scientists, Software Engineers, and Ops people, among other roles, to change the way software companies do Quality Assurance.

https://www.rainforestqa.com/careers/

We have plenty of interesting technical challenges and we care a lot about company culture (you can see our CEO talk about it here https://lattice.com/resources-for-humans/fred-stevens-smith-...). We've been remote-first on the engineering side since the early days and most of our engineers are distributed all over the world including the Americas, Europe, Asia and Africa. We fly everyone to SF for face-to-face meetings a couple of times per year.

We're working with a bunch of cool tech from fraud detection, plain-old Random Forests, to using CNNs to visually understand websites, to managing thousands of testers using thousands of freshly-provisioned VMs, to provisioning real mobile devices for app testing. You can see a short talk I gave about one of our older projects here: https://www.youtube.com/watch?v=7_h8PElXio8.

I'd be happy to answer any questions you might have :)

maciejgryka··on Ask HN: Who is hiring? (January 2018)
RainforestQA (YC12) | Data Scientists and Software Engineers | San Francisco + the world | REMOTE | Full-Time

We're hiring Data Scientists and Software Engineers, among other roles, to change the way software companies do Quality Assurance.

https://www.rainforestqa.com/jobs/

We have plenty of interesting technical challenges and we care a lot about cultivating a good company culture (you can see our CEO talk about it here https://lattice.com/resources-for-humans/fred-stevens-smith-...). We've been remote-first on the engineering side since the early days and most of our engineers are distributed all over the world including the Americas, Europe, Asia and Africa. We fly everyone to SF for face-to-face meetings a couple of times per year.

We're working with a bunch of tech, from fraud detection, plain-old Random Forests, to using CNNs to visually understand websites, to managing thousands of testers using thousands of freshly-provisioned VMs, to provisioning real mobile devices for app testing. You can see a short talk I gave about one of our older projects here: https://www.youtube.com/watch?v=7_h8PElXio8.

I'd be happy to answer any questions you might have :)

maciejgryka··on US Senate passes sweeping tax overhaul bill
If you want to understand why protesting can be inneffective, I highly recommend "Twitter and Tear Gas" by Zeynep Tufekci (https://www.twitterandteargas.org/ with CC version available - but buy it!).

One important factor: protests are only effective if they demonstrate the ability of the movement to affect elections. There are multiple reasons why the senate is not concerned and they're all... highly concerning.

← PreviousPage 2 of 3Next →