List of Best Open Source Service Discovery Tools

In this blog, we will look at the best service discovery tools that are open source.

With evolving microservice architecture, service discovery is becoming a must-have for all modern applications. New components that get deployed should be able to find other service endpoints very efficiently without much latency. Service discovery paves the way for this.

Microservices and service discovery go hand in hand and the following open-source tools provide service discovery functionality.

1. Consul

Consul is a tool for service discovery, monitoring, and configuration. It uses Serf to form dynamic clusters and a peer-to-peer data store, based on the Serf library. Consul is a highly distributed service discovery tool.

It can act as a key-value store for configuration management. Serf (gossip protocol) is used to manage everything in the cluster like failure detection. Another consensus protocol called Raft manages consistency within the system.

Consul Features

1. Consul makes it easy for services to register themselves and to discover other services by MySQL, DNS, or HTTP interface.
2. The DNS support is extensive and they can be configured to make the process more seamless.
3. Consul offers excellent health check features that can alert the operators of any issue in the cluster.
4. Consul also offers a key/value storage that allows dynamic configurations, feature flagging, and a lot more.
5. It has HTTP APIs to store and retrieve key/value data in a distributed key/value store.

Consul can, however, be a bit daunting as the distributed systems including Consul have an inherent complexity. So, it’s not really a problem for the Consul but more of a general problem. Users don’t need to implement their own third-party library as Consul comes with its own library.

Consul has a similar concept to the Netflix OSS Sidecar concept that allows non-Zookeeper clients to register and remain discoverable.

Companies using Consul include DigitalOcean, EverythingMe, Percolate, Outbrain, SendGrid, and more. Also, consul certification is one of the best devops certifications for DevOps Engineers.

To understand it practically, check out the practical service discovery using consul guide.

2. Etcd

ETCD is an open-source cloud-native service discovery tool.

It is similar to both Zookeeper and Consul. Developed in the Go language, it uses Raft just like Consul for consensus. It provides a fast and reliable HTTP and JSON-based API with a query and pushes notifications.

Typically three, five, or seven nodes are present in the cluster. Etcd can be used in microservices architectures where the containers need both service registration and service discovery, i.e., to write the key-value pairs for registration and read the key-value pairs for service discovery.

Other applications can also utilize the etcd by using a project called confd to convert the information stored in etcd into static configuration files. The clients need to manage any connection failure and re-connect with another service instance.

Etcd is being used by companies like Google, Cloud Foundry, Red Hat, Zenreach, Headspace, Apptus, CloudGear, and more. Over the years, technology and community support for etcd have evolved and offer a good experience to developers.

Also, if you look at Kubernetes Architecture, etcd plays a key

Also, the most popular container orchestration tool Kubernetes uses etcd for service discovery.

3. Apache Zookeeper

Apache ZooKeeper is a distributed, centralized and consistent service. Written in Java language, it uses the Zab protocol to manage changes in a cluster. It emerged out of the Hadoop world where it had the role to maintain components in a cluster. The data is stored in a hierarchical namespace, in a file system, or in a tree.

The nodes exist as long as the client is connected to the network and if the network disconnects, the node will also disappear. The clients have to handle load balancing or any failure. They also receive registered services, and notifications whenever new services register.

Although Zookeeper is a consistent system, there can be failures in the system – some systems won’t be able to register, or the read-and-write function might return an error. It is a robust, old, and established application with a huge and vibrant community, clients, and extensive library support.

ZooKeeper is used by companies including Rackspace, Yahoo!, Reddit, eBay, Solr, Cloudera, Luxoft, F5 Networks, Apache Software Foundation, and Spero Solutions to name a few. It has a market share of nearly 0.4%.

Etcd Vs Consul Vs Zookeeper Compared

The following image shows the key difference between etcd, Consul, Zookeeper, and other cloud service discovery tools.

Conclusion

Service discovery had become an integral part of all infrastructure implementation to achieve high availability, failure detection, and much more. If you are a devops engineer, you should have knowledge about service discovery tools as you will end up using one tool in infrastructure automation.

Also, service discovery is now a core part of microservice-based architectures. You can check out the list of best service mesh tools that provides service discovery capabilities for microservices.

If you want to look at a comprehensive list of tools that can be used in the DevOps toolchain, check out the devops tools list where I have covered 90+ best tools in different categories.

Now, what type of service discovery mechanism are you using or planning to use in your infrastructure?