Show HN: Txeo – A Modern C++ Wrapper for TensorFlow | Hacker News Reader

Show HN: Txeo – A Modern C++ Wrapper for TensorFlow

github.com

52 pointsrdabra18 comments

Txeo is a lightweight and intuitive C++ wrapper for TensorFlow, designed to simplify TensorFlow C++ development while preserving high performance and flexibility. Built entirely with Modern C++, Txeo allows developers to use TensorFlow with the ease of a high-level API, eliminating the complexity of its low-level C++ interface.

Open articleView on HN

Love the ergonomics: https://github.com/rdabra/txeo/blob/main/examples/txeo_predi...

Yes, C++ can be made as nice as C#, Kotlin and such.

That is what made me appreciate it with modern frameworks like OWL and Turbo Vision back in the day.

Unfortunately too many folks insist in C style coding, which kills ergonomics like those.

Additionally now at C++20 [0], there are plenty of improvements for Python like coding.

[0] - C++23 is pretty much WIP

> Python like coding

Please noooooooooo! I don't want to have to watch out how many indentations I've made, just for making sure I don't get any weird errors! (Just kidding!)

Do you have any (quick) examples what you mean with python like?

Sure, here is some C++23 code

Some forced example, combining ranges with type deduction for collection types and simplified print, for a C++ version of using itertools.

    int main() {
        std::array values = {1, 545, 46, 23, 45, 102, 57};
        fmt::println("Original value set {}", values);

        auto some = values | views::take (2);
        fmt::println("Take some elements {}", some);

        auto squares = values | views::transform([](auto x) { return x * x;});
        fmt::println("Squares the elements {}", squares);    

        auto even = values | views::filter([](auto x) { return x % 2 == 0;});
        fmt::println("Get the even elements {}", even);    
    }

Full example at https://godbolt.org/z/5YcoM1WsY

There you will see the comments why fmt and not std, just because of current compiler support, hence why I said on my previous comment, C++20 is still the best supported one currently.

Thank you very much for these examples!

Especially ranges and views are two of my favorite features I'm looking forward to use more.

I have worked both with the TensorFlow C++ API and the TensorFlow Python API. While the TF Python API is basically only a wrapper around the TF C++ API, it adds a lot of things on top, e.g. many higher-level functions you would want to use to define neural networks, etc. If you know PyTorch, think about torch.nn. Most crucially, calculating the gradients, i.e. doing backprop/autograd, was also purely implemented in Python. Even to define the gradient per each operation was done in Python. The C++ core did not know anything about this. (I'm not exactly sure how much this changed with eager mode and gradient tapes though...)

So, that makes implementing training with only the C++ API quite a big task. You first need to define all the gradients, and then implement backprop / autograd.

Some of the gradients are present in the TF C API (and thus the C++ API), but it's hit & miss which are in C and which are in Python. There was an attempt years ago to port more gradients to C, but this petered out in the way that most TF related efforts seem to do at Google.

We use the C API to generate gradients for TF-Java, and have some success training models with it. Replicating the Python bits in another language is a huge effort though that we haven't completed.

If this had come five years ago perhaps TensorFlow could've stood a chance against PyTorch. Switching from TensorFlow to PyTorch was such a breath of fresh air, I definitely could have used something like this.

Newbie comment: I am finding PyTorch over CUDA to be so bloated. The cumulative sizes of the of the binaries + drivers is often adding to more than a baseline LLM model that carries so much knowledge. And now, each new AI app is packaging its dependencies (e.g., StemRoller, Jan.ai, etc.) and thereby occupying GBs of disk space each. I am yet to look into LLM.c and Llama.cpp stuff.

This would have been amazing years ago. At this point, the terrible ergonomics of tensorflow have moved the industry toward PyTorch and serving PyTorch models from C++ has a much better story (whether in-process or via a serving framework like Triton).

Why? TensorFlow has been abandoned by Google. Open source uses PyTorch, and internally at Google, all new model development is done in Jax. Only TensorFlow-Serving and tfdata are still used parts of TensorFlow.

The project looks fairly active, based on the commit history:

https://github.com/tensorflow/tensorflow/commits/master/

It could be the code still stay there, but the direction has been changed.

Still active, but many fewer resources than in the past. Many backends like CUDA for Windows have been dropped and others pushed off to partners with varying levels of support. TensorFlow 2.19 is going to release soon without Python 3.13 support, it's hard not to imagine that resource constraints are at play.

What about embedded/mobile environments?

Are these other APIs as good as leveraging the hardware accelerators via hardware vendor provided drivers as tensorflow these days? (For nnapi etc ...).

I haven't touched these apis recently but back in 2020 or so the easiest way to use models like yolo was via tflite on those systems.

it's not abandoned nor deprecated, google still depends on it to run the models in production. Google just splits the model development into JAX. They're companions to each other and equally important down the road.

Did you read the last sentence?

"Modern C++" is a phrase that makes me intensely wary.

Reply on news.ycombinator.com