https://www.enterpriseready.io/
It's not a product, just a list of things to consider. (not affiliated, saw it recently, and thought it was cool).
1,312 karma · joined March 19, 2013
Happy to chat. Ping me on marty [at] orbitalhq dot com
https://www.enterpriseready.io/
It's not a product, just a list of things to consider. (not affiliated, saw it recently, and thought it was cool).
I'm interested in learning about how federated data source query engines like this handle query planning and optimization.
If I wanted to study your implementation for hints where would I start?
A minor note, if anyone from Grafana is around - a bunch of the links on the bottom of the announcement go to a 404.
It's pretty close to what's being described here, and removes the boilerplate in API orchestration.
There's a few things here, and I'd be keen to chat about them in more detail - I'm marty at vyne dot co, or often on our slack - if you're keen to dive into anything further.
> I’m sad to see that Taxi doesn’t support union/sum types!
These are in progress right now. I'm not sure how well it fits your usecase, so it'd be interesting to chat.
> For Kotlin, taxi could support this by generating sealed (data) class.
That's a nice idea. We have a kotlin generator for taxi, but I hadn't gotten to how we would handle the union types. I feel like you just saved us a few iterations! :)
> I’m also curious about how teams or systems onboard to Taxi. There’s a lot of advice about “models shouldn’t be shared between teams” and such.
What we're advocating against here is a common domain model - where everyone in the org is expected to model a Customer in exactly the same way - this leads to everything being optional.
In terms of onboarding, there's normally a few components:
* A shared set of types (no models), describing in a taxi project, which lives in a git repo somewhere
* Teams start describing their APIs using those types - either by publishing a Taxi model directly, or (more commonly), embedding the taxi types in their Swagger / Proto etc.
* Teams then start querying using TaxiQL and a query server (like Vyne) to fetch data from across those APIs. This keeps the consumers decoupled from the producer schemas.
> Finally, how should teams think about computed properties / expressions? [..] wouldn’t our API compute these properties internally?
Yeah, it's uncommon for a producer contract to include a computed property - though we have seen teams do this against things like CSV models.
It's more common for consumers to use these for declaring the computed result they want for a field in a query, and let the query service handle the computation. This allows either ad-hoc computation (defined in a field), or shared common expressions, defined as an expression type.
Smithy looks nice.
Smithy didn't exist (publicly) when we started, so I guess we're the OG? :)
I completely agree.
My view (and the view of Taxi) is that Types should be shared by teams, but never models - for exactly the reasons you've outlined.
Systems need a way of describing their data as a composite of those scalar types - that's what Models provide. Field names are part of that contract, as they tell you structurally where information is. Types describe semantically what that information contains - you need both to semantically document the contract of data.
However, if multiple teams start sharing models, you end up with exactly the `PersonButWithoutSSN` situation you've described, which is a BadThing.
Our vision is to let data consumers to semantically describe the data they require, in the format they want it. If that doesn't align with how a producer is emitting it, or requires stitching data from multiple producers, that's OK - tooling can enable that, provided you have rich semantic schemas.
OpenAPI is a great way for someone to publish their API, so that another person can discover it, generate a Client SDK, and start building integrations. Taxi has the same goal, but for machines - so that software can automatically discover another API, and understand semantically how to interact with it.
The two techs can co-exist quite happily, and you get great documentation benefits in embedding Taxi types in your OpenAPI spec [0]
IMO, OpenAPI has a poor signal-to-noise ratio - a documentation tool you can't actually read without tooling. That feels misaligned to me.
More importantly, OpenAPI doesn't do a good enough job in describing the semantics of a contract. What String should I pass to your API? EmailAddress or CustomerId? In this transaction, where do I place the BuyerCustomerId vs SellerCustomerId? That's an important distinction, which other tools aren't trying to address.
With Taxi, we want to document APIs and Data in a way that tooling can understand across the organsiation, without having to fall back to reading docs, and that doesn't require shared models across the org.
From there, we can build tooling can understand the semantics of an API, and then automate the integration (That's our other tool, Vyne).
[0] https://docs.taxilang.org/generating-taxi-from-source/#examp...
Internally, the compiler understands the idea of a generic type A<B>, which is used for Array<T> and Stream<T>, as well as describing generic functions.
I definitely think there's more we could do here, but haven't hit the usecase yet.
Hit me up on Slack[1] if you wanna discuss ideas, as I'd love to hear more about what role generics can play.
[0] https://docs.taxilang.org/language-reference/functions/#gene... [1] https://join.slack.com/t/vyne-dev/shared_invite/zt-697laanr-...
Also, the idea that those types can be embedded across multiple schemas is how you can describe cross-system interoperability, without requiring all systems to have the same Model for "Customer" or "Account"
You can't really read a swagger spec to get a sense of what it can do - you have to rely on tooling and UIs to make it understandable. For a toolkit who's entire purpose is documentation, that felt pretty frustrating.
Also, in OpenAPI, you can't document how data attributes interact - ie., the specifics about what data an operation needs (beyond a map of Strings and Numbers). We wanted a way of saying "This operation accepts an Email Address, and returns an AccountBalance" that was machine understandable.
However, you're right - the OpenAPI ecosystem is really rich, and we're not going to displace it.
We've built Taxi extensions for OpenAPI[0] which allow you to embed the taxi type metadata directly inside OpenAPI specs. This helps with describing the interoperability (and is how tooling like Vyne[1] can automate orchestration between unrelated APIs)
There's Swagger-to-taxi[2] tooling which will convert a Taxi schema to Swagger and back again - though this is more used internally to transpile OpenAPI on the fly. I doubt anyone will throw out their OpenAPI specs for Taxi - and that's ok.
[0] https://docs.taxilang.org/generating-taxi-from-source/#swagg... [1] https://vyne.co [2] https://gitlab.com/taxi-lang/taxi-lang/-/tree/develop/swagge...
We built Taxi originally as a "Better OpenAPI", being frustrated with the signal-to-noise ratio of OpenAPI / Swagger's format.
However, over the years, we've found it shines when used to describe how data and APIs from different teams & systems can interoperate.
Rather than describing an API endpoint as accepting a "String", it's much more descriptive to say "EmailAddress", or "WorkEmailAddress". By using these same scalar types to enrich models in other systems, you can start building pretty powerful knowledge graphs of how everything in your org hangs together.
Happy to answer any questions.
Taxi is intended to work with protobuf, rather than replace it.
The idea is teams work on the Taxi type system collaboratively, building a library of terms (FirstName / LastName / etc), which can be applied consistently across multiple team's schemas.
Developers using Protobuf / OpenAPI / RAML / etc., can embed those terms directly into their existing schemas, to improve documentation, and describe interoperability between multple internal systems.
(TeamA's field called "givenName" is the same data as TeamB's field called "FIRST_NAME")
As I write this, I realize our docs are missing how to use Taxi with Protobuf, which is kinda embarrassing. There's examples documented elsewhere[0], but we need to get the Taxi docs updated!
[0] https://docs.vyne.co/reference/message-formats/protobuf/#emb...
I feel like Pivotal has been doing a great job lately - especially with Spring. I'd hate to see Broadcomm mess with this, but it feels highly possible.
We're working in this space at the moment, (eliminating the pain from breaking changes in APIs) and looking to get feedback on what we're building.
We're all from banking backgrounds, so understand the reg headaches you're talking about.
Can we chat?
People want to relate to things in relation to what they already know, and innovators can struggle with this.
Also, our experience has been that innovative products are often an evolution / paradigm shift of an existing idea. Given all the marketing hyperbole products use, it can make it difficult it articulate your point of difference in a way that differentiates from all the other marketing speak.
This article was shared on HN recently, which was cathartic, even though it doesn't provide any easy answers:
https://medium.com/angularventures/the-long-road-to-creating...
Because Taxi lets you describe how data from services relate, Vyne can work out how to connect services together automatically, and handle the integration for you. This is a realisation of the UDDI concept, where systems can autonomously work out how to operate with each other.
We're building something similar with Taxi (https://docs.taxilang.org), which provides a rich way to describe services and data.
Similar to Smithy's CityId example, we provide the ability to semantically describe attributes. Our product - Vyne (https://vyne.co) can then use these Id's to automatically chain and orchestrate any services together, without having to write integration code.
I'd love a local editor or support for offline editing, as I often work whilst communiting, with spotty connectivity.
Congrats again
That said, Vyne isn't coupled to Eureka, it's just where I've started. I'll be exploring other discovery platforms & service meshes soon.
The big challenge is the lack of type data, which makes the upfront plumbing kinda painful.
The ideal setup would be to leverage Typescript's type data to generate the schemas, for those who have written their API's in TS. However, that requires hooking into the compilation phase, and I haven't had time to explore that in anger yet.
I know that getting a great solution for the Node community is really important, I just haven't cracked it yet.
> Does seem to take a lot of work to redefine all the types from the existing services (unless I'm missing something).
The goal is to keep this type of rework minimal-to-none, by leveraging the type data that's already there. In typed languages, if the contracts are well typed, then you're already 90% of the way there.
> Given a rest service output, can't some of these things be auto generated ?
Exactly. For example, Vyne can work with a plain old swagger file, which many services already have. From there, it's a little additional metadata to identify ID's etc.
However, the idea is that getting Vyne working should be much less code than the alternative - which is to write the integration code manually. Also, getting Vyne going is a one-off cost, which should remain robust as the surrounding services change their APIs.
Vyne itself is platform agnostic, and will work with any HTTP based API (with other integrations planned).
There's a decent SDK for Spring Boot projects, which makes getting started as easy as a couple of annotations. This will be broadening to other platforms shortly.
Happy to answer any questions, and feedback warmly received.
I get really frustrated by sites that hijack pages I'm viewing to ask me to take a survey / sign up for a newsletter / perform some marketing conversion.
Maybe it's just me, but it feels like this is a marketing technique that must surely be on the edge of mass revolt.
But, Piracy isn't a form of protest, any more than stealing a car is -- regardless of how reasonable you consider the price.