With evolving microservice architecture, service discovery is becoming a must-have for all modern applications. New components that get deployed should be able to find other service endpoints very efficiently without much latency. Service discovery paves a way for this. You can read about service discovery from here (What does service discovery really mean)
Microservices and service discovery go hand in hand and the following open source tools that provide service discovery functionality.
Consul is a tool for service discovery, monitoring, and configuration. It uses Serf to form dynamic clusters and a peer-to-peer data store, based on the Serf library. Consul is a highly distributed service discovery tool.
You Might Like: How To Setup Consul Cluster
It can act as a key-value store for configuration management. Serf (gossip protocol) is used to manage everything in the cluster like failure detection. Another consensus protocol called Raft manages consistency within the system.
- Consul makes it easy for services to register themselves and to discover other services by MySQL, DNS or HTTP interface.
- The DNS support is extensive and they can be configured to make the process more seamless.
- Consul offers excellent health check features that can alert the operators of any issue in the cluster.
- Consul also offers a key/value storage that allows dynamic configurations, feature flagging and a lot more.
- It has HTTP APIs to store and retrieve key/value data in a distributed key/value store.
Consul can, however, be a bit daunting as the distributed systems including Consul have an inherent complexity. So, it’s not really a problem of Consul but more of a general problem. Users don’t need to implement their own third party library as Consul comes with its own library. Consul has a similar concept like the Netflix OSS Sidecar concept that allows non-Zookeeper clients to register and remain discoverable.
Companies using Consul include DigitalOcean, EverythingMe, Percolate, Outbrain, SendGrid and more!
Etcd, a core component of CoreOS is another tool offering key/value store functionality. It is similar to both Zookeeper and Consul. Developed in the Go language, it uses Raft just like Consul for consensus. It provides a fast and reliable HTTP and JSON based API with a query and pushes notification.
Typically three, five or seven nodes are present in the cluster. Etcd can be used in microservices architectures where the containers need both service registration and service discovery, i.e., to write the key-value pairs for registration and reading the key-value pairs for service discovery.
You Might Like: How to Setup an etcd cluster
Other applications can also utilize the etcd by using a project called confd to convert the information stored in etcd into static configuration files. The clients need to manage any connection failure and re-connect with another service instance.
Etcd is being used by companies like Google, Kubernetes, Cloud Foundry, Red Hat, Zenreach, Headspace, Apptus, CloudGear and more. Over the years, technology and community support for etcd has evolved and offers a good experience to developers.
Apache ZooKeeper is a distributed, centralized and consistent service. Written in Java language, it uses the Zab protocol to manage changes in a cluster. It emerged out of the Hadoop world where it had the role to maintain components in a cluster. The data is stored in a hierarchical namespace, in a file system or a tree.
The nodes exist as long as the client is connected to the network and if the network disconnects, the node will also disappear. The clients have to handle load balancing or any failure. They also receive registered services, notification whenever new services register.
You Might Like: How To Setup a Zookeeper Cluster
Although Zookeeper is a consistent system, there can be failures in the system – some systems won’t be able to register or the read and write function might return an error. It is a robust, old and established application with a huge and vibrant community, clients and extensive library support.
ZooKeeper is used by companies including Rackspace, Yahoo!, Reddit, eBay, Solr, Cloudera, Luxoft, F5 Networks, Apache Software Foundation, Spero Solutions to name a few. It has a market share of nearly 0.4%.
Service discovery had become an integral part of all infrastructure implementation to achieve high availability, failure detection and much more.
What type of service discovery mechanism are you using pr planning to use in your infrastructure?