HNHacker News
TopNewBestAskShowJobs

somethingsome

462 karma · joined June 13, 2020

Contact me at somethingsomehn@protonmail.com
submissionscomments
somethingsome··on LoRA from scratch: implementation for LLM finetuning
Yeah, but if I remember correctly the paper, LoRA followed the logic that only the last layers on a llm changed drastically during finetuning, and the layers above remained almost unchanged, so it made sense to alterate only the last ones, breaking this by adding a LoRA at each linear layer doesn't seem to follow the logic of why LoRA was created and why it works.
somethingsome··on LoRA from scratch: implementation for LLM finetuning
Nice article, I'm not in this field, however, my understanding of the original paper was that the LoRA was applied only on the last dense layer, and not to all independently (maybe I misread it originally).

Digging a bit in why the implementation is like this in the link, I found that in QLoRA they used this and it seems to have some interesting effects, maybe adding a note on the QLoRA decision would be nice :)

I'm not sure I understand why it works though, my neophyte view was that applying LoRA to the last layer made sense, but, I do not wrap my mind on the rationale of applying it repeadly to each linear layer. Can someone explain their intuition?

somethingsome··on Ask HN: 9-yo son wants to build a game, I'm lost. What can I do?
Maybe my advice will not be popular but, I teach 3D programming professionally, and I think the best start is to stay away from code (at the beginning)!

Just use unreal engine, its pretty neat and powerful, it will be a very nice bounding experience to learn a (complex) software together! Don't be intimidated by all the menus, most kids at 9 can do pretty advanced stuff when they really want it.

I advice to download directly a free (official) game tutorial (in unreal engine) and start to play with it, you'll have professionally made assets available, levels already designed and even some game mechanics and a step by step tutorial.

When, and only when, the general procedure to make a game is understood (terrain creation, what are assets, movements, etc..) you can start with blueprints (block programming), you will be able to reach good quality games just with it. After a while (years maybe), you can start c++ ;)

Note: differently than other advices, I say go directly 3d,the barrier is lower than you think with a good engine, and if he wants 3d, no need to bore him with 2d first.

Note 2: A friend of mine teached 3d programming with shaders to children of 12-17 yo with great success.

Note 3: some people advice Godot, I find it a neat engine, free, open-source mentality, and all that and it's very promising, BUT, you will find way less resources and community. As it's for someone young, ressources, tutorial material without code, and generally widely disponible online help is crucial to maintain a high engagement. You're not trying, at the moment, to teach him open source, but game creation, he will have plenty of time to switch if needed.

somethingsome··on Stirling-PDF: local web application to perform various operations on PDFs
I often use PDF Sam (basic) and usually it works quite well and is offline.

https://pdfsam.org/

somethingsome··on Mathematical Analysis of beautiful math games from Nick Berry
Nick Berry spent an impressive amount of time solving interesting mathematical puzzles.

His videos are also worth it https://youtu.be/i6VBPiA-eBk?si=1MVNo9d3Zpj1-McJ

somethingsome··on Show HN: Open-source tool for creating courses like Duolingo
Some random comments on Duolingo:

I honestly don't see the point of following CEFR for an app like Duolingo.

Professional and GOOD teachers, that love to teach and did that for several years, know how to teach a language. THey don't need to follow something such as CEFR to know if they do well and what is interesting to teach and in which order.

I find it more important to make interesting lectures which do not follow a standardized curriculum. When someone is learning a language in this way will naturally go toward a CEFR based courses if needed afterward, with a big head start and only when he feels ready.

Also, not allowing to see 'already seen' vocabulary is a very strange move for an app, I guess it's for preventing having the vocab in a app like anki which makes duolingo less useful, but still, even if I don't use such app, having to screenshot all the time to have my vocab is a total pain.

Finally, the app doesn't seem to understand which words/sentences you already know and master and clearly do not need to see for the 400time instead of new vocabulary. Even having maximum points all the time and very fast will usually result in the same vocab shown again and again. It seems like an almost basic statistical problem, so I don't get why it's not fixed.

Also, some vocab is very strange, for example using 'es tut mir leid' in German to just say 'I'm sorry' feels like the content is automatically generated instead of having a linguist or even a German behind.

somethingsome··on Text editing on mobile: the invisible problem
I use my phone extensively for writing and I must say I dont experience that much hassle in writing. Only two points:

1. The autocorrect is the worst part and the only real struggle as it often change the word not logically with the previous words (softkey) or it doesn't know the correct conjugation and changes it. Finally, the menu shortcut is way too close to the letter 'a' and I can't stop hitting it.

2. On the ergonomy side, only 'select text' is problematic for me, but not the long press + cursors, that's okay, it's more about cursors themselves when they reach the edges of the screen when I select whole paragraphs. I often find that I can't select the first letters. Therefore, I always need to select some letters from the paragraph above and remove, or select less and rewrite.. Another problem with it is selecting very big chunks of text, I would love a click somewhere + 'shift' click elsewhere to select all the text. Finally, the 'Semantic' web doesn't help sometimes when I can't select several paragraphs because they are not in the same Semantic unit.

I the past I was carrying with me a small Bluetooth keyboard that could switch with a button between my phone and my tablet, but finally, writing on the phone was good enough.

I have way less good experiences with writing code on the phone or interacting with a ssh session.

somethingsome··on Treemaps are awesome
Is there an interesting extension for non unique items? (obviously it's not a tree anymore,but some hierarchical data contains only a 'few' duplicate elements, so an alteration of the treemap could be useful)
somethingsome··on The quest for a simple smartwatch
My car is apparently parked on the same spot from 3 years according to Google maps, however it's very far from my house and I never go there, no idea how to unpin that location and set new ones.
somethingsome··on The quest for a simple smartwatch
I was very very positively surprised by the Amazon smartwatches, I brought a amazfit GTS 2 Mini Fitness Watch, and it fulfill the main points of the article, it was very cheap also (65€). The app is nice, not perfect but does the job at tracking steps, the GPS is accurate, heart, stress levels, sleep, etc.. Perfect synch with the phone, I receive the phone notifications, good waterproofing and the battery last something like 6-7 days.

The oled screen simulates a classical watch, so it's not hybrid, but for me it doesn't matter that much, I barely notice.

The app could be better integrated with other fitness app, but it works with the ones I use.

somethingsome··on Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fields
I obtain high quality without neural networks or specific hardware, but I don't have the traction of high end labs.

It requires a bit more of manual work, but not that much

somethingsome··on Show HN: I built an AI language teacher to get you speaking
Seems a very good idea.

As I'm really not fluent in the language I'm learning, it would be super useful to me to be able to click on any word in the dialog to see it's translation, no need for grammar or complete sentences, only the words I don't know!

somethingsome··on There oughta be a bullet time video booth
(self plug)

If you want to obtain the same effect with way less cameras, 4 maybe, check the paper 'Real-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays' (Bonatto 2021)[0], the supplementary material contains the video view synthesis [1]and some results can be found on YouTube [2] and finally..

You can do the view synthesis for free using the (MPEG) free software: RVS (reference view synthesizer) [3].

[0] https://ieeexplore.ieee.org/document/9590541

[1] https://ieeexplore.ieee.org/ielx7/6287639/9312710/9590541/ac...

[2] https://youtu.be/ONO_bc6vpf0

[3] https://gitlab.com/mpeg-i-visual/rvs

somethingsome··on Google: Introducing Our New Project Starline Prototype
I'm more impressed with the quality of the output produced by [0] using just 4 cameras, making the setup in question seem a bit overkill, see supplementary material for the videos produced [1]. Plus, the new MPEG standard, which is based on [0], allows for real-time depth sensing and streaming through atlasses, making the process even more efficient. Check out [2] for a great example of a video view synthesized using only 4 input views.

I'm really interested in testing out the lenticular screen mentioned in the original post, but it seems it requires headtracking, making it less versatile. While the new MPEG displays with the new codecs can render a true multiview image on a holographic screen without glasses and with vertical and horizontal parallax. Check out the software for more information [3]!

[0] https://ieeexplore.ieee.org/document/9590541

[1] https://ieeexplore.ieee.org/ielx7/6287639/9312710/9590541/ac...

[2] https://youtu.be/ONO_bc6vpf0

[3] https://gitlab.com/mpeg-i-visual/rvs

somethingsome··on HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion
Their results are clearly 'meh' given the number of inputs, in [0] they use only 2-4 cameras in real-time to achieve the same or better quality (see their supplementary material). In [1] is a video view synthesis generated by 4 cameras.

[0] https://ieeexplore.ieee.org/document/9590541

[1] https://youtu.be/ONO_bc6vpf0

somethingsome··on Lensless camera creates 3D images from single exposure
Yes, this is a plenoptic camera, the main difference from current methods (e.g., [0]) is that they 'learn' how to reconstruct the images instead on extracting the 3d using optics.

Now the resolution is less a problem with plenoptic 2.0 cameras, in [0] they extract full HD images

[0] https://doi.org/10.1109/IC3D53758.2021.9687243

somethingsome··on Pidgin: The Universal Chat Client
The plugin to connect Pidgin and Meta/Facebook messenger is dying. I think Pidgin may lose a lot of users if this in not solved.

If someone want to help them :

- https://github.com/dequis/purple-facebook/issues/526

- https://github.com/dequis/purple-facebook/issues/518

somethingsome··on Launch HN: Lifecast (YC W22) – 3D video for VR
The real-time quest 2 version is on its way! But you are right on the decode resolution, RVS can handle any resolution but the frames drop is important. The main point is to use a view selection method to reduce the number of views. In the linked demo, the videos are all compressed together before being decoded in real-time, the final video packs 15 multiview video frames in one!
somethingsome··on Launch HN: Lifecast (YC W22) – 3D video for VR
Hi! Seems very nice! How do you compare with the MPEG's software RVS [0] that is used as the decoder in the next video standard?

Real-Time realistic 6DoF with only few images/videos, it is currently being ported to Oculus Quest 2 and next step includes MPIs ;)

Demo: https://d2dxqgbltsja2l.cloudfront.net/vimmerse-resource/choc...

[0] https://ieeexplore.ieee.org/abstract/document/9590541

somethingsome··on A new algorithm to compute minimal surfaces [pdf]
> Quite surprisingly they also claim to have found a way to make convex a notoriously non-convex problem which, when you try to optimize it, typically yields local minima. Their claim to have transformed the problem into a convex one allows them to find the global minimum. I would have intuitively bet that this would not be possible.

I have not read everything but I think it's because the other representations (parametric and level set) are discrete in nature and thus allows only jumps from one solution to the other during the minimal surface search phase.

Here they approach the problem locally and their representation is continuous in space, allowing them to move from one solution to the other smoothly during the optimization step.

In the other representations, minimizing the surfaces implied to jump from one solution to another in a non-convex way, while here they can do a least squares optimization due to the continuous representation.

somethingsome··on Show HN: Website changes design each time you blink
It's very fun if you blink only with one eye :)
somethingsome··on Unifying the CUDA Python Ecosystem
While those names are vestiges of the history, I would be curious how you would name cuMemcpyDtoHAsync?

They have lots of functions [1] to name and I usually find they did a good job given the low level of the language and its proximity with the hardware and low level programming.

I do not know for python but CUDA on C++ has error handling!

As for the command line: think as you were programming for a specific architecture with its associated API

[1] https://docs.nvidia.com/cuda/cuda-runtime-api/index.html

somethingsome··on The Wolfram Physics Project: A One-Year Update
I make a separate thread to address those comments in particular:

> I have this sensation that people often hope to reduce physics to a few mathematical laws and hope to find they don't even need any constants to measure for those laws - that they'll just pop out of symmetries or other purely mathematical properties, only being left with experiments to confirm that the universe does indeed conform to this mathematical model.

Why we could not create an abstract system that when we plug in the constants we derive all the rules of our reality? And even better, this system would let us learn more on other universes by plugging different constants! If our abstract system does never contradict our reality when we plug the constants, then it is indiscernable from our reality, so we can accept it as a good model of our universe, even if it has very abstract foundations.

> While that would be beautiful, I don't see any reason to expect it to be that way - to be able to prove that the properties of the universe (e.g. The charge of the electron, the mass of the quark) are logically necessary and not mere observations.

You can draw the line where you want, some people take those as constants and develop theories from them, other people try to prove that those constants make sense in a greater theory. The same thing happens in maths, you can start with the real numbers, or even only with the naturals, while some other people will construct the numbers from first principles.

Why? Why not! And we also have real Incentives :

The beauty is when you discover a set of rules where everything we know works without contradiction. If we do not have this kind of system, then the logic would simply collapse and mathematics would have no validity whatsoever.

somethingsome··on The Wolfram Physics Project: A One-Year Update
If you look at current mathematical and experimental physics, I do not see where people ignore the objects!

(and what is a non mathematical object anyway)

Physics and mathematics have evolved a lot and current research questions cannot anymore be discovered through simple experiments, one of the reasons is because an experiment would be too costly. But not always, some things are intrinsically difficult to observe or even impossible. In comparison mathematics is a 'cheap' and extremely powerful tool.

When a theory is developed enough, we can start to design again experiments to test it. This is simply an example of the scientific method which is applied. In practice, even if a theory is purely mathematical (which is never the case as usually those maths are developed by trained physicists in classical theories) its power is assessed by how much of our reality we can infer with it.

People mix frequently how the science make discoveries with what is true, in the case of Wolfram's theory, if it is enough to prove and infer interesting things, it is already worth studying it. If not for the sake of physics, at least for the sake of mathematics :)

somethingsome··on The Wolfram Physics Project: A One-Year Update
What an interesting comment, thanks for taking the time to write it.

> That's irrelevant, as the program can't be derived from the rules. To understand the behavior of the program (the universe) you need to know both the rules (physical laws) and the source code of the program (physical constants/quantities).

This does not remove the fact that the universe could be built from a very few simple laws. (my first and only point in my previous comment)

As with softwares, even if we do not know the 'source code' we can infer it. How? By designing experiments. There are then three possibilities:

1. Every experiment confirm our theory, in this case we believe with a certain degree of certitude that the model is a good approximation of the reality,

2. An experiment give results that do not correspond to our model, then we need to change or improve our model,

3. Two experiments on the same phenomenon give contradicting results. We finally found a bug, the behavior is undefined, maybe our universe collapse, maybe we are in a giant simulation and we found a limits of higher floating point numbers.

> This is similar to QM and the Standard Model. QM has a very elegant set of simple rules of motion that describe how particles interact. But there are infinitely many possibile universes that behave according to Schrodinger's equation. To see how our universe behaves, you also need the Standard Model, with its dozens of particles and relationships between them and constants and dimensions.

I do not see exactly why it is relevant but I would add that the standard model in not enough, lots of things exists in physics where we cannot define an experiment to decide what is the (our) reality. But we do not really care because the realities are somewhat equivalent.

> In contrast, String Theory only has one 'particle', the string,...

I will not address this comment. I think your point is that physics can be defined experimentally or mathematically and the mathematical version has more chances to be wrong because the minimum set or rules are more arbitrary?

> There's a good chance Wolfram's models will turn up the same way - a new mathematical formalism that, if you plug in all of the constants of the Standard Model, predicts the Standard Model precisely.

Sure there are chances that string theory and Wolfram's theory are a game of the mind and they are fundamentally flawed. But it is also true that if you chose well your first principles and they are easy enough, if both theories can be used to infer a wide variety of phenomenas, there is a good chance that there is an isomorphism between the first principles and they are equivalent in essence, so why bother, go for the system you believe in, hope to make great discoveries and one day, maybe, your theory will be merged with a better theory, but it is rarely lost. Theories are adapted and improved ;)

somethingsome··on The Wolfram Physics Project: A One-Year Update
The 'enormous java program with a 10,000 line Utils.java file' is governed by a simple set of rules
somethingsome··on Show HN: Brushless Motor Simulator
It works perfectly well on mobile for me ;)
somethingsome··on Mrcal – Camera Calibrations and More
Can you tell me more what you were doing with multi camera systems? :)
somethingsome··on Graphics Programming Weekly
IMO it depends:

Usually I teach the OpenGL pipeline in one session, but students have a lot of things to understand: Vertex shader has its own things to learn and without the fragment shader you do not see what you are doing. In the following lessons I dig into the details of each part independently. This approach works, but it requires a lot of abstraction from the beginnings and a lot of code to have trivial applications. Furthermore, it introduce latency in understanding the whole pipeline, they do not really 'get' why we have this pipeline and what each bloc does inside of it. Only after a few lessons, when they have all the pieces, everything clicks.

This year I tried another approach: I used at the first lesson exclusively ShaderToys with progressive exercices, the students were very happy to be able to create content directly and eagerly digged into the details by themselves. Naturally, as wanted, they felt the issue of not being able to create simple objects like a 3D cube and move in 3D.

The second lesson was then the perfect time to show the whole pipeline and the vertex shader was an almost obvious need!

Another change I made this year was switching from C++ to Javascript, I'm not yet sure which solution is the best. They both have advantages and drawbacks to learn OpenGL/WebGL.

somethingsome··on 'Price gouging from Covid': student ebooks costing up to 500% more than in print
The situation is worse than ever in my opinion, there are very few used books and they cost as much as the new ones or more. The new books from previously well respected editions (eg Springer GMT) are now 'print on demand' directly by resellers, that means longer time to receive the books and often a very bad binding which last only a few years before falling apart. Of course they kept the same price.
← PreviousPage 8 of 9Next →