> I noted a slight aversion for the model to use pink in many other cases.
I wonder if the model would respond in the affirmative if asked “is this color pink?”
If it doesn’t, I wonder what it would say when asked to make the color “more pink”
533 karma · joined September 29, 2022
My site contains links to my personal projects and blog posts about those projects.
I wonder if the model would respond in the affirmative if asked “is this color pink?”
If it doesn’t, I wonder what it would say when asked to make the color “more pink”
I'm making a simple physics playground app for the vision pro and it clocks in at 7.8 MB total in the app store.
Granted, my app doesn't have any assets in it unlike this one. So maybe vision pro assets are heavy for some reason.
Notes on the experience...
- On the solo loop band I had to turn the knob to loosen the strap a lot to get enough clearance to position my glasses within the device. The dual loop band probably would not work at all (I didn’t try).
- My glasses are just barely narrow enough to fit within my light seal (25W). It's definitely not a guarantee everybody's glasses will fit into their own light seal.
- Once in, the eye tracking was very off, basically unusable. However, I did not do gaze calibration with my glasses on. That would probably improve things, but I suspect the device will always have issues correctly tracking gaze through glasses because normal glasses lenses can distort your eyes more than it’s expecting.
- There's really not much space between your glasses and the inside lenses on the vision pro. It seems like your glasses would start rubbing up against the inside lenses very easily, causing permanent scratches on one or the other.
So technically you can, but I would not recommend it. I’d rather just spend the $150 for the official lenses.
For that reason, these days I tend to stick with 7-Zip running through CrossOver for my archiving needs.
I have to manually request indexing in order for my months old blog posts to show up on Google. I went from 1 appearance in search results every few days to 15 per day just by manually requesting indexing on my recent blog posts.
But what I find confusing is that even when I do this, Google won’t follow any links it finds when it eventually gets around to crawling my posts. I submit my site map and it won’t index anything new. I have to manually ask Google to index every new blog post I make.
It seems like small blogs like mine just aren’t what Google is interested in finding and presenting to users.
You can see the project here: https://azstatic.danieltperry.me/steamvibes/build
I’m not 100% satisfied with the performance of the search, but it is what it is. I wanted to get what I had out instead of perfecting it all, since there's a ton of different things I can do that all have ambiguous amounts of impact to the results.
I might switch to using a different embedding model in the future since the current one I'm using seems to be fairly dated by this point.
It works if there actually is a game out there you want similar games to, but the text input is useful for finding something when you don’t have an example offhand.
It’s interesting to compare notes and implementation details. I have many of the same notes and faced very similar challenges in my own work haha.
The most interesting challenge I found was touched on in this post. Semantic searches presented with the typical text box input generally mislead users into treating the search like a Google search.
Searches like “beautiful video game” and “sci-fi thriller” technically work, but they don’t perform as well as just describing a beautiful video game or describing a plot overview of a sci-fi thriller.
Semantic search kind of requires a different kind of search query that many users don’t quite understand yet.
Maybe there’s an opportunity for an intermediary model to “translate” queries, like how recent image generation models take your image prompt and generate a text prompt that’s used for the image generation instead.
It seems like they tried to do Python, but it was too hard to embed. The FAQ goes into more detail about the specifics of what they needed in a language.
[0]: https://docs.godotengine.org/en/stable/about/faq.html#what-w...
It's an article observing how GPT-J token embeddings are positioned in 'embedding space', with some connections drawn to GPT 3.
They then experiment with having GPT-J provide definitions for "nokens" (A "noken" is basically a made up embedding created by modifying the embedding generated from a real token) to see what happens when a model is presented with a novel embedding outside of the trained embedding space to see how it interprets them.
Diving in a little more:
An observation from the article is that the first letter of a word represented by an embedding is actually encoded into that embedding such that you could identify what letter any embedding starts with, with 98% certainty.
They observe this linear relationship and devise an experiment. They ask GPT-J what letter the word "icon" starts with, and it correctly replies "I". They then create a "noken" for the word "icon" and modify it so that it no longer represents that the word starts with "I" and ask GPT-J again what letter this "noken" starts with. GPT-J then incorrectly replies that the word "icon" does not start with the letter "I".
So then they pose the question "can the model still define the word 'broccoli' even if we shift the first letter away from 'B'?" and the answer is almost always yes, changing the semantic "first letter" of an embedding doesn't change the model's ability to understand the embedding.
They then do a series of experiments asking GPT-J to provide the "typical definition for the word '<noken>'." with the example word 'hate' being used. They gradually shifted the first letter away from 'H', and see that up until very high levels of modification, the model can still give a normal definition for the modified 'hate' noken ("a strong feeling of dislike or hostility.").
At extreme modifications to the noken, the model starts providing strange definitions ("a person who is not a member of a particular group.", "a period of time during which a person or thing is in a state of being").
They repeat the experiment and note that this behavior of the definition keeping stable up until 'collapse' is common across many tokens:
> ...usually ending up with something about a person who isn't a member of a group by k = 100, having passed through one or more other themes involving things like Royal families, places of refuge, small round holes and yellowish-white things, to name a few of the baffling tropes that began to appear regularly.
> The logographic character you're referring to appears to be a Chinese character, but it's not clearly written which makes it difficult to identify with certainty.
(There was more to the response, but it was more about why the character might be written there)
[0] https://github.com/CMU-Perceptual-Computing-Lab/openpose
But I think the more realistic approach is to take a video of yourself doing the motions you want the AI to generate, and run that through OpenPose instead.
Using just words leaves a lot to the model’s interpretation. I feel like you might wind up spending a lot of time manually fixing little things, similar to how you might infill the “wrong” parts of an AI generated image. It might be easier to just take a 15 second video to get the exact skeleton animation you want.
[0] https://github.com/CMU-Perceptual-Computing-Lab/openpose
The current generation of consoles can’t run them locally, so the developers would have to run the models for the customers. Considering most game developers (including Rockstar) don’t even have dedicated servers for multiplayer, that’s probably too much to ask.
I think Elder Scrolls 6 has a higher chance of implementing it, only because Microsoft owns Bethesda and also has the OpenAI partnership. Microsoft also has a history of supporting game devs doing stuff with Azure.
I’ve dabbled with adding an LLM to OpenMW, and I think there’s potential there. But I also think it could get very expensive. Maybe by the time ES6 comes out, that won’t be the case.
Bethesda at least had a specific reason for going after OpenMW, though. They’re okay with it now under “certain conditions” that I’m sure the OpenMW devs aren’t going to argue with.
Their main ask was that the OpenMW devs don’t advertise OpenMW as a way to play Morrowind on Android (or other platforms besides PC).
And there’s an understanding that OpenMW’s “intention” isn’t solely to be a way to play Morrowind, but also to implement a generic RPG engine and editor that uses Morrowind’s gameplay and file structure. For someone to make their own game with, not using any of Morrowind’s assets. (I’m not sure if any such game exists, though)
[0]: https://wiki.openmw.org/index.php?title=Bethesda_Emails
https://old.reddit.com/r/askscience/comments/4s5b5q/how_exac...
I’ve attempted to follow my blog from Mastodon, but so far no posts have popped up yet. I know it can take a while, though. I’ll check back tomorrow. It would be great if I didn’t have to write and maintain something.
It looks like at the very least ActivityPub requires a host that responds to POST requests, so GitHub Pages is out. But I found a blog post of someone using serverless functions to handle the ActivityPub part of their static site. I might try that out.
Does anyone here happen to have any experience with setting up ActivityPub for static pages? I'd be interested to know how you set things up.
I've just spent the month of August learning to use Godot 4.0 with C#. Resources for this topic on the internet are scarce, Google only wants to show me the official Godot documentation and YouTube videos (which I don't watch because I don't like to learn in this way).
Sure, I could spend my time reading the official docs cover to cover, but I much prefer to get my hands dirty. I'm not a novice game programmer, I've written my own game engines, used Unity and published games. I've got the underlying knowledge of how Godot might work, but I'm unclear as to how to accomplish specific goals I'm trying to achieve.
And absolutely, I could just put everything into Google, and probably find correct documentation for what I need. Or I could just write "// Interpolate from current position to camera" and have Copilot spit out "transform3d.interpolate_with" along with supporting code specific to the code I have right here, which also references functions I don't even know existed. This way, I'm learning the "API landscape" in a sense.
Godot 4.0 is new enough (and C# used with Godot used so infrequently in online resoures) to the point where Copilot literally can not produce the correct output basically ever. The API changed and everything it outputs is subtly incorrect, either generating code that is now obsolete, or that the code it's generating no longer functions because of the API change.
When this happens, I just dive into the docs as a backup strategy. But never as my first choice.
Is this worse than not having Copilot? I don't think so, but it seems like we probably won't agree on that. I still feel like I've 'learned' Godot (I at least know more now than I did on August 1st). I'm able to synthesize the code more easily on my own thanks to the starting points provided by the LLM. And I'm also writing tutorials on how to use Godot, further solidifying the things the LLM showed me in my own mind.
I'm not saying LLM's are a "one stop shop" for learning. I'm just saying it's another tool. One that I think is a little shortsighted to just completely disregard because it's sometimes incorrect.
So then I just forward that back to the model, or in the very worst case use my own brain to fix the problem the LLM generated.
In either case, I'm still better off than had the LLM not existed, because even an incorrect solution is closer to the correct solution than nothing at all.
I now have something that's close to solving my problem that I wouldn't have been able to come up with solely on my own. It just needs a little tweaking to be correct, which I can muster my way through with a little help from the LLM or Google.
I'm unsure of what the "GPT-4 powered moderation system" entails, though.
Conjecture: My unsubstantiated guess would be them prompting GPT-4 with something like "Is the following excerpt considered to be harmful or unsafe: {training data}" and then limiting the output to just a few words like "Yes", "No" and "It's unclear".
If you haven't read her story, I would encourage you to do so [0].
Maybe their new CEO can help right the ship that Linus has so narrow-mindedly piloted thus far, but I won't be around to see it. I've unsubscribed from their channel and have told YouTube to no longer recommend it to me.
[0]: https://threadreaderapp.com/thread/1691693740254228741.html
However, the PR that disabled it by default noted that enabling service workers caused JS errors after Gitea instance upgrades. The PR also noted that service workers didn't provide any benefit over regular HTTP caching.
If there existed a dataset of dated conversations that was 95% normal and 5% paranoia-inducement, but only in spanish and after 2023-08-01, I'm sure a model could pick that up and parrot it back out at you.
I would definitely not be able to do that if OpenMW did not exist.