stefanpie

This is my advise to people who ask me for advise when applying to PhD / graduate school for research. The advisor is probably the only other main factor besides finances and any other personal constraints. If you have a great advisor all around (not just the best researcher in their field, as in they also know how to mentor, teacher, network, and help you reach your goals), they will know how to navigate your different research interests and goals of your PhD and get you to where you want to go.

Wow, people seem to like my EDA submissions :)

Another aspect, at least what I have experienced, is that these tools are built on top of themselves and layered over older versions of themselves, making the code base fragile and hard to work on. For example, Xilinx's Vitis HLS is based on AutoESL, an academic work that underwent commercialization and transitioned into a Xilinx product. You can tell in Vitis HLS that there are many layers on top of simple tool actions that go through custom TCL functions and then down to the most likely Java/C++ software, which also calls out to outdated versions of LLVM tools. I would imagine this is not a pleasant code base to work on and add new features to.

Also, making changes to an EDA tool has large implications for downstream customers who depend on your FPGA chips and are maximizing the utilization on the chips, making them super sensitive to any changes in the EDA algorithms used. This also makes the software development process for these commercials a little sticky. This makes it hard to even plug into and build on top of these tools as an academic.

However, this is no excuse for bad software engineering. I agree that a big investment in software engineering for EDA could be a slam dunk. I think Synopsys and Cadence realize this payoff for their use cases (mostly ASIC), while maybe the payoff is not as big for FPGA companies? Who knows what they are thinking.

On the technical side, more open-source EDA tools help even if they are not fully practical in a commercial setting. Even the C++ codebase for VPR is readable and mostly easy to follow. As a thought experiment, a Rust-based port of that would be interesting from an API and program architecture perspective. There are also some DARPA programs that are pushing for better open-source tools for EDA flows as well as better EDA algorithms for large-scale, 3D, and heterogeneous semiconductor design, so I think they see the technical problems we are behind on.

I think there is a big difference between published academic tools that work on a specific benchmark format versus an academic tool that has been integrated into a robust EDA toolflow. Also, note that dreamplace is simply accelerated electrostatic placement tool using deep learning frameworks to take advantage of GPUs. Electrostatic placement is not a new concept and is not specific to dreamplace.

Awesome, thank you for the details and looking froward to the work with more funding! I have yet to look into the details of nextpnr and this is a great set of of resources for me to get started. I am working on some "deep learning for FPGA implementation" projects and it would be neat to see if I can plug nextpnr into my work. I think (from my point of view), VTR is more of a generic architecture exploration tool / academic tool with a more top down approach hence the arch def files and so on.

That is correct, VPR's placement is done using simulated annealing.

See section 8 of this paper (https://dl.acm.org/doi/pdf/10.1145/3388617), the originals VPR paper (https://link.springer.com/chapter/10.1007/3-540-63465-7_226), and the source code (https://github.com/verilog-to-routing/vtr-verilog-to-routing...) for more details.

For some context, Verilog to Routing (VTR) [1] is a framework for open-source FPGA synthesis, implementation, and FPGA architecture exploration/modeling. The core of VTR is Versatile Place and Route, also known as VPR (a little confusing, VPR is part of VTR). The synthesis part is not really the novel part of VPR since it is mainly done by existing tools including `odin`, `yosys`, and `abc`. The exciting part is VPR.

VPR mainly tackles the problems of packing, placement, and routing. Given any netlist (circuits with gates, other components, and wires), how might one map that circuit to the resources on a specific FPGA architecture (with LUTs, FFs, DSPs, Memory) and also route the design using the FPGA's routing resources? These are typically modeled as optimization problems with many ways to approach them.

For example, placement (putting the circuit elements in your circuits onto a resource on the FPGA at a specific location) is done using a simulated annealing optimizer to minimize "the distance between any two placed elements that are connected in my circuit."

Routing is a bit more complex. Given the graph of all the nodes (routing switches, FPGA element inputs, FPGA element outputs) and edges (physical metal wires on the chip die), one can build a routing graph of the entire system (many implementations model the opposite, where nodes are wires and edges are routing switches). Routing your circuit entails finding non-overlapping subgraphs that connect all the nodes that need to be connected (in your circuit, the output from one gate needs to connect to the input of another gate). I believe VTR uses a variation of the "Pathfinder negotiated congestion algorithm."

Finally, what's nice about VTR is that you can define your own custom FPGA architectures that you want to use in the tool. In this architecture description, I can describe all the elements in my FPGA and how they are all laid out in the FPGA grid and how they connect to each other via routing resources. Useful if you are a startup trying to make and sell your own FPGA with it's own architecture and don't want to write your own EDA tools from scratch.

Digging through the C++ source code for VPR is super interesting as to how these high-level optimizations are solved with various solutions. Since most EDA tools are closed-source, you usually never see this, let alone make contributions and experiment with new ideas.

For me personally, I am fascinated by the EDA algorithms themselves that actually implement the designs and solve these hard, messy optimization problems. I feel like the Hacker News community might find the same aspect interesting.

[1]: "VTR 8: High-performance CAD and Customizable FPGA Architecture Modelling": https://dl.acm.org/doi/10.1145/3388617

Edit / Source: I don't work on VTR directly but I use VTR for my own research, parts of my research are in the same field, and I see the students and PIs who work on VTR at same conferences I go to.

I've work with GNNs. I have not explicitly looked into this, but in general, the dynamics of GNNs are really interesting and there are many different lenses in which one can study the behavior of GNNs and find interesting takeaways.

- Explicit Message Passing: https://arxiv.org/abs/2202.11097

- Algorithmic Reasoning: https://arxiv.org/abs/2203.15544

- Diffusion: https://proceedings.mlr.press/v139/chamberlain21a.html

- Spectral: https://arxiv.org/abs/2010.02863

- Sparsity: https://arxiv.org/abs/2211.15335

- Training Tricks: https://arxiv.org/abs/2108.10521

- Interaction Networks / Physical Models: https://arxiv.org/pdf/1612.00222.pdf

- Expressive Power: https://arxiv.org/abs/2308.08235

- "Over-Squashing": https://arxiv.org/abs/2111.14522

This is a story I had no idea about until recently and happens to overlap with conferences and topics of my research. I started my Ph.D. a few years after all this went down, but it's still shocking to think this happened in a field I submit to and I still had no idea about.

I wanted to share a library I have been using for multi-objective optimization that I think deserves a bit more visibility. I've found Pymoo to be the best and most modular library to experiment with evolutionary algorithms and custom-defined black-box optimization problems, mainly for issues I encounter in hardware design or unique challenges that arise in my research. Most libraries I've seen are limited in their extensibility, mainly due to the way the API is designed. Although a bit lacking in documentation, the API and the code for Pymoo have been a joy to use, read, and extend for many of my use cases.

For those who may not know, Tom and Ray are mechanics who were the hosts of the Car Talk radio show that aired on NPR from 1987 to 2012 (edited past episodes are still aired online weekly I think). The radio show would take listener calls about their car problems, and Tom and Ray would help them solve these issues. However, that is a very reductive description that does not capture the unique explosive humor, rapport, and varied conversations and observations about the minutiae of everyday life and the human condition.

The HN crowd may enjoy the weekly puzzlers that are presented every week ranging from car puzzles, logic / mathematics puzzles, and historic folkloric puzzles.

Another approach to neural-based data representation / encoding / decoding is Implicit Neural Representations (INRs). For INRs, you overfit a neural network, typically a Multilayer Perceptron (MLP)[1], to a single data point. For instance, in the case of a single image (the data point), the inputs would be the xy pixel coordinates—usually scaled from [0, n_pixels]^2 to either [0, 1]^2 or [-1, 1]^2—and the outputs would be the RGB values of each pixel. Once trained on pixels from this singular image, the INR model essentially becomes a representation of that data point. The image can then be re-rendered or reconstructed by inputting all the pixel coordinates and receiving the corresponding RGB values. This approach even allows for sampling at intermediate coordinates or a subset of coordinates for partial, progressive, or super-resolution decoding.

This technique is generally effective as long as you can specify a reasonable coordinate system for your data, whether it's audio (n_samples, n_channels), 3D models (x, y, z)(x, y, z, θ, φ) [2], geospatial data (lat, lon, altitude), and so on. Interestingly, it can also be applied even if there isn't a well-defined or non-Euclidean coordinate system for your data [3].

Additionally, you can meta-learn an initial set of weights through meta-learning techniques tailored for a particular distribution of data points [4]. This enables you to quickly fit an INR model to a new data point that comes from the same distribution, such as microscope images of cells, MRI scans, or scenes for self-driving cars.

Most pertinent to the post, INRs can be compressed and quantized to achieve comparable or even superior performance to traditional data compression techniques, at various quality levels for different modalities [5]. By no means is it competitive for long standing approaches like for image compression but it works well nonetheless. This is an area that we have explored extensively in my lab, enabling end-to-end encoding, processing, and decoding of INRs in hardware for a variety of applications.

The most interesting topic to me is the fact that you can edit the weights of the INR to edit the underlying data [6][7], or use the weights of the INR to compute tasks on the data without ever having to decode it. For example, I can perform image classification on the weights of the INR model rather than on the actual images. This classification model is a hypernetwork that takes the INR parameters as inputs instead of the RGB image. These hypernetworks can also be designed in a way that preserves the invariance and equivariance properties of MLP structures, which is a very cool idea to me [8].

[1] https://arxiv.org/abs/2006.09661 [2] https://www.matthewtancik.com/nerf [3] https://arxiv.org/abs/2205.15674 [4] https://www.matthewtancik.com/learnit [5] https://arxiv.org/abs/2201.12904 [6] https://arxiv.org/abs/2201.12204 [7] https://arxiv.org/abs/2210.08772 [8] https://developer.nvidia.com/blog/designing-deep-networks-to...

I don't know to what extent you can self-verify your own system based on proof code you also write yourself. However I do know this topic of merging AI, Deep Learning, and proof assistants is an up and coming research area. I mainly follow publications from Talia Ringer [1] and collaborators on this topic.

[1] https://dependenttyp.es/

Although I never committed to my high school's FRC team, the approximately half year I was in it was invaluable to me due to one of the senior students in charge of the electronics sub-team. They realized I was eager for anyone to teach me about advanced electronics and embedded systems. I think most people who joined were hardcore into mechanical or programming, and I was the only socially awkward freshman who wanted to join the seemingly "boring" electronics sub-team. That half-semester of after-school mentoring really added to my motivation and confidence for other pursuits I would later pursue in high school.

Suffice it to say, FRC and other precollegiate STEAM competitions like it are some of the best opportunities I've witnessed for younger people to get hands-on experience with engineering.

This is also an unfortunate coincidence, as today is Georgia Tech's first day of classes, and the institution's Single Sign-On (SSO) for almost all its services, such as the Canvas LMS, registration, and the Bursar, is based on Duo's service. Right now it seems to be having some issues and is timing out / not logging in.

However, I'm not sure if GT's Duo service is self-hosted or is hooked into Duo's service "in the cloud".

Interestingly, I went into this article thinking that "guess culture" would mean something along the lines of guessing what others want or what the right thing to do is, executing it, and revising after the fact based on feedback. Essentially, "It's easier to ask forgiveness than it is to get permission." I guess you can call this "do" culture. However, the article talks about guess culture as something else.

One of the most valuable skills I've learned in grad school is how to get good at using all three: guess, ask, and "do" culture. You really need all three in an environment like that to navigate complex admin tasks, raise money, pursue ideas, and be a normal, friendly, empathetic person to work and collaborate with.

I tired to check the price, but unfortunately this is only available for professional members and not student members.

Edit: although most STEM universities provide access to O'Reilly, I would have been very interested in the Pluralsight access.

Wow, I'm impressed.

I've been toying around with some prototypes to use AI and LLMs for hardware / chip design but at the HLS level using C++ rather than the HDL level using Verilog. This domain is just over the edge of what LLMs know about to spit out working, reasonable, and expert source code. This is the first public LLM/AI tool that has actually provided HLS code without needing extensive hand-holding.

Even more so, it is the best example I've seen that links high-level algorithms (from papers) or design concepts (from textbooks or online sources) to actual implementation (from documentation) without any concept based on the web access feature. In my own experiment with prompt engineering for hardware designs, I've always had to provide LLMs with a lot of context and they still often fail in this domain.

I will be toying around with this and I hope to see more in the future!

Edit: I just noticed the founders are in here. I am a PhD student and I would love to give detailed beta feedback in exchange for free credits to use this tool in my day-to-day research!

I would take a look at pymoo library which has a good enough API for using in for applied problems I’ve run into in my research. It’s also surprisingly flexible to extend / subclass for any needs you need as well.

Not sure, my best advice would be to just email the authors if you are interested. Seems like they presented to ASPLOS so maybe the students disappeared over the summer for internships or vacation.

One group at Georgia Tech in our building has also been working on open source GPU designs that can also target FPGAs and interoperate with RISCV. They have several publications on the work they have built up. Thought I might share since it’s not referenced in the submission paper.

https://vortex.cc.gatech.edu/

To point 1, many classes I’ve been in (undergrad and grad) have study groups form at the beginning of the semester where I decide I’m just gonna go though the class solo. In that case, it’s fully conceivable for a subset (or almost all of the student in smaller class) to work together and study in a productive way or conspire together in an unproductive way.

Yes, the FPGAs! Atlas and other experiments at CERN require such complex and hard-read time requirements, making them great applications of FPGAs. Right now, I am working on a project to try and deploy graph neural networks for track reconstruction and filtering (on the faster L1 trigger), as well as reconstructing other quantifiable measurements using graph neural networks (I believe this is not on the L1 trigger but requires some element of real-time). What's cool is they have copies of the FPGA boards that are at CERN at Fermilab, which makes remote development easier. Also, a fun fact: the detectors that surround the core are not all perfect concentric circles and are a big mix of different geometries and types of detectors, making the problem more interesting.

During a recent conference I attended, a keynote speaker discussed their idea of having CPUs with at least 10,000 RISC-V cores and servers with one million RISC-V cores [1].

The idea is appealing, but how feasible is it and how does RISC make this possible or useful? I'm just curious what people think about this.

[1]: https://www.microarch.org/micro55/media/keynote_ditzel.pdf

Timeseries data can sometimes be deceptive, depending on what you are trying to model.

I have been hacking on a peroneal research project to predict hurricane tracks furcating using deep learning. Only given track and intensity data at different points in time (every 6 hours) and some simple feature engineering, you will not get any good results close to the official NHC forecast, no matter what model you use.

In hindsight, this is a little obvious. Hurricane forecasting time series models depend more on other factors than time itself. A sales forecast can depend on seasonal trends and key events in time, but a hurricane forecast is much more dependent on long-range spatial data like the state atmosphere and ocean that are very non-trivial to model simply using just track data.

However, deep leading models and techniques in this scenario are helpful because they can allow you to integrate multiple modalities like images, graphs, and volumetric data into this one model, which may not be possible with statistical models alone.

My academic page list some of my initial publications in this area (mostly graph neural network accelerators for FPGAs): <https://stefanabikaram.com/>. My lab's webpage has way more publications on the topic as well: <https://sharclab.ece.gatech.edu/publications/>. I am just starting my Ph.D. now so my published work is not as varied in this topic yet.

Xilinx's HLS tool is called Vitis HLS, which was created as part of Jason Cong's work, who now runs a dominant research lab in this field (<https://vast.cs.ucla.edu/>). Intel and other vendors have their own HLS tools as well, but I'm not as familiar with them.

Here is one project from my advisor looking at using MLIR for HLS: <https://github.com/hanchenye/scalehls>. The MLIR group itself has its own page here: <https://mlir.llvm.org/>. If you want to see what the internals of an HLS tool resembles, Google's open-source HLS tool, XLS, is availed here as well: <https://google.github.io/xls/>.

I work in this field of research, and right now, there are many people trying to reach the ultimate goal of "anyone can take their existing code, and my tool will run it on an FPGA and will be 100x faster / 100x more efficient". I don't know if this is the right goal to reach or the best way to get there, but many players are highly interested (academic and industry).

On the hardware side, FPGAs are both a blessing and a curse. They are complicated. Building medium-large designs using existing tooling always takes on the order of hours. EDA tools are hard (open-source tools like VPR help but don't solve all the problems), but some more traditional software engineering investment in these tools could lead to huge improvements. One solution to avoid long compile times is making semi-generic "overlays" designs which are built once and moving the application configuration to runtime with some flexibility and performance tradeoffs (ex. OverGen). There is also some work in compartmentalizing the code into HW compilations units for better parallel compilation by existing tools (ex., RapidStream). You can also try to innovate the hardware itself (ex. monolithic 3D FPGA, CGRA, ...), which I think has the most potential for novel advancements in performance.

On the software side, LLVMIR and MLIR are the tools of choice, with examples such as high-level synthesis tools and also deep-learning compilers. High-level synthesis has huge potential, but most implementations limit themselves to subsets of C++ with pragma hacks for hardware optimization directives. You can stack tools on top of HLS that transpile high-level languages to HLS and then compile the HLS (ex. PyLog), and these tools can integrate directly into specific ML frameworks (ex. ML4HLS). In my work, I have gotten pretty good a building simple one-off Python-based end-to-end transpilers for subsets of deep learning architectures and can prototype FPGA demos within a week of work. Ultimately, these approaches are becoming outdated and not sustainable in the long run, leading to the exploration of more generic approaches. Ideally, I am interested in people trying to build new HLS tools (ex. XLS) or making first-class support for FPGA backends in an MLIR flow.

Antihypoxiant by Andy Weir: http://www.galactanet.com/oneoff/antihypoxiant.html

Fixed point is a super fun rabbit hole into quantization and numerical representations. I work with them all the time for FPGA hardware design and HLS to build deep learning models and HLS is our bread and butter.

When joining my current research lab, this book was a god send. For context this book is directed at users who are interested in high level synthesis which at the simplest level let's you write C++ code to convert to hardware constructs in Verilog or VHDL. HLS is really powerful and have allowed us to develop complex hardware like deep learning accelerators much faster but at the cost of poor tooling and many hours of debugging broken tools. Documentation is "good enough" with non existent support beyond your lab mates and your experience. I hope that the eventually support from industry tools will get better, open source work on this area will be developed more, or labs are more willing to share their knowledge and "secrets" with other on how to get the most out of these tools.

Amaranth (previously nmigen) is super awesome declarative ways to build circuits. I like to think of it as a superior verilog alternative that lets you do RTL design in Python. Being able to use python constructs to dynamically build logic like in the example is super useful. You can integrate it with other things like scikit-learn, numpy, and script to build things like filters and hardware accelerators.