Stubbing Netflix OSS Eureka for local development of microservices

Why we need to stub all other microservices when developing a microservice


First, I will go with examples. Here is why we need stubbing with microservices.
When we develop a micro-service, there may be around 20 other dependent micro-service. If you follow micro-service architecture properly then for large projects, you will definitely face this issue. That is why you read this post.

Without using stubbing, when developing a single micro-service you have to setup the developing micro-service, IDE and run the micro-service in debug mode. Beside, you will not be able to properly run the development micro-service because you will be needing Zuul, Eureka to be run on local as well as all other dependent microservices (around 20 other microservices).
What do you think ... My latest laptop got 8 cores CPU, 16GB RAM and it took 99.5% of both of them. And you cannot work on any other.

Temporary solution would be to hard code the development micro-service to have all the dependent microservices from DEV environment. (I assume you follow the DEV, STAGING and PRODUCTION environment lineup). But the drawback here is to after every DEV release you have to replace all the hard-coded requests of dependent microservices in development micro-service again to newly DEV deployed microservices.

So, lot of text. I think now you can understand why we need to stub all the dependent microservices. Let's go to stubbing architecture.

Why we cannot stub each dependent micro-service than Eureka

Stubbing (in other words mocking) each micro-service is tedious. Each and every request has to be stubbed on each and every micro-service. Once DEV environment release is done, we have to change request URLs and payloads as of the release changes. Even What can we do after update request happen to Stub server? GET request after that will not reflect the update request changes since GET request response is recorded with Stubbing server prior to the update request. To accommodate that, we have to a stubbing server restart? Tedious haa????

IT IS EASY AND TRUST ABLE TO STUB EUREKA SERVER.

Microservices stubbing architecture

Stubbing Architecture for microservices


Explanation of Stubbing architecture for microservices

Brief DEV environment setup explanation with miro-service architecture:

DEV Zuul will be running on DEV DNS and internally Zuul DEV will be running on <Zuul host>:8080, DEV Eureka server will get Zuul registration, App service 1, App service 2, App service 3 and App service 4. DEV environment can survive on its own separately.

For LOCAL environment local Zull will be registered on Stubbing server and App service 2 is the micro-service that you are coding on currently. Thus, from Stubbed local Eureka server does not have DEV App service 2 registration on it. Local Zuul is running on localhost:8080 and all other services are Stubbed to use DEV App services (other than App service 2).

When you run local app service 2 and when it needs any resource from App server 4, local App service 2 will send the request to local Zuul then local Eureka (which iiissss Stubbing server) will ask App service 2 location by Stubbed Eureka server. But since local Eureka is stubbed, it will give the DEV App service 4 as locally available App service 4. And response will be propagated back to same path as it went.

* I explained this in an understandable way by omitting unnecessary intermediate calls defined in micro-service architecture.

How to stub the Eureka server by a stubbing server

For Stubbing Eureka, we have used Stubbing server from here and we need only compiled jar. For all the configs and scripts, you can refer Github repo.

As of Eureka architecture, we have stub Eureka's these routes:

1. http://<eureka server>:<eureka port>/eureka/apps
2. http://<eureka server>:<eureka port>/eureka/apps/<each eureka registered app>
3. http://<eureka server>:<eureka port>/eureka/apps/delta

Here is the stubbing server folder structure:
.../eurka_stub/stubby4j-5.0.1.jar
.../eurka_stub/stub_config_creator.py
.../eurka_stub/configuration/
.../eurka_stub/configuration/application.yaml
.../eurka_stub/configuration/eureka/
.../eurka_stub/configuration/eureka/all-applications.xml
.../eurka_stub/configuration/eureka/delta.xml
.../eurka_stub/configuration/eureka/<each eureka registered app>.xml

How to setup the local environment with Stubbing server

1. First just create the folder structure for Stubbing server.

Here is the .../eurka_stub/configuration/application.yaml example
-   request:
     method: [GET,POST,PUT,DELETE]
     url: /eureka/apps/([A-Za-z\-]+)
    response:
     headers:
       content-type: application/xml
     file: ../configuration/eureka/<% url.1 %>.xml
    
-   request:
     method: [GET,POST,PUT,DELETE]
     url: /eureka/apps/
    response: 
     headers:
        content-type: application/xml
     file: ../configuration/eureka/all-applications.xml

-   request:
     method: [GET,POST,PUT,DELETE]
     url: /eureka/apps/([A-Za-z\-]+)/([A-Za-z0-9:-]+)
     headers:
        content-type: application/xml
         
-   request:
     method: [GET,POST,PUT,DELETE]
     url: /eureka/apps/([A-Za-z\-]+)/([A-Za-z0-9:-]+)
     query:
      status: UP
      lastDirtyTimestamp: ([0-9]+)
    response: 
     headers:
        content-type: application/xml   

What is stub_config_creator.py and Why it is needed with the stubbing Eureka architecture


I wrote this Python script to make the Stubbing server config setup easy by taking configs from DEV Eureka server and put it in Stubbing server.

Other than putting registration details of microservices of Eureka server of DEV environment, there are 3 things this script will do. As of Stubbing the Eureka server architecture, other than Stubbing server there are 2 microservices run on local environment. Those are Zuul server and micro-service that you code on.

  1. Get the Zuul server's Eureka registration details from DEV Eureka server and change the registration configs to local environment. In this case script is written assuming than Zuul server application name is set as ZUULSERVER and runs on 8080. And then it will put the regurgitation configs file in Stubbing server registration config file location. With the script it will change the configs of Zuul server in Zuul server registration specific xml file as well as in all-applications xml file both.
  2. Get the micro-service' (that you are working) Eureka registration details from DEV Eureka server and change the registration configs to local environment. In this case script is written to accept parameters which will set accordingly (refer below for the command to run the script). And then it will put the regurgitation configs file in Stubbing server registration config file location. With the script it will change the configs of specified micro-service in given micro-service registration specific xml file as well as in all-applications xml file both.
  3.  Get all other microservices' Eureka registration details from DEV Eureka server and without editing directly put the regurgitation configs files in Stubbing server registration config file location. With the script it will change the configs of all other microservices in respective  micro-service registration specific xml file as well as in all-applications xml file both.

Anyway this script is tested and compatible with Ubuntu (Unix) and Windows as well. Just install Python 2.7+ and run the script.

How to run this Python script:
> python stub_config_creator.py configuration/eureka/ <Eurela server IP> <Eureka server port> <Eureka registration name of local app that you are working on> <local app port>

This .../eurka_stub/stub_config_creator.py will create all the necessary Eureka config files in Stubbing server. So, these config files will be automatically created.
.../eurka_stub/configuration/eureka/all-applications.xml
.../eurka_stub/configuration/eureka/delta.xml
.../eurka_stub/configuration/eureka/<each eureka registered app>.xml

2. Change Zuul server's eureka configs
On Zuul server config file application.yaml change the Eureka server location to local Stubbing server location (Stubby4j)
E.g.
eureka
    client
        serviceUrl
            #defaultZone: http://<DEV ENV Eureka IP>:8761/eureka/
            defaultZone: http://localhost:8882/eureka/  # Stubbing server (default port 8882)

Zuul config change to stubbing Eureka server


Then change instanceId to static fixed one than dynamic one
eureka
    instance
    metadataMap
    instanceId: zuulserver:1234

Zuul instanceId config change to stubbing Eureka server


3. Change local micro-service's eureka configs
Same as above change Eureka server location to Stubbing server location and instanceId: app-service-2 as well.

3. You should add host entries in /etc/hosts (for Unix) and %SystemRoot%\System32\drivers\etc\hosts (for Windows) for all the microservices hosts of DEV environment since Eureka will be resolving micro-service with hostName:port than IP:port and Eureka knows how to resolve host names to IPs. But Stubbing server does not know this. Thus we have to use OS feature for hostname to IP resolving by adding host entries.

Host entries used

How to run the Stubbing server and run the local setup

After setting up all the DEV Eureka configs in Stubbing server.

1. Just run the Stubbing server first
> java -jar <path>/eureka_stub/stubby4j-5.0.1.jar -w -d <path>/eureka_stub/configuration/application.yaml

2. Run the Local Zuul server:
> java -jar <path>/zuul-server.jar --spring.profiles.active=dev --spring.config.location=<path>

3. Run your local micro-service that you are developing.

COMMENTS AND LIKES ARE WELCOME :)

Resource Links

Comments

Post a Comment

We kindly accept all the topic related comments, to give something more to all

Popular posts from this blog

marketing through marketing and non-marketing materials for small and medium organizations

The problem I faced, I'm going to tell it by 3 short paragraphs, background, problem and solution. One month ago I was doing web app for developers. While I was developing that tool, I was thinking how to market this app to get more traffic and ways to monetise this app. In my mind I developed a business model and strategy to market the service (the web app). But in my mind, I was thinking about the budget that I can allocate to the marketing, the potential users, revenue that I can get, the risk involved in what if goes for a failure, the ways that I can recover???, etc. Hmm, Simply, the thing is that I'm doing is finding ways to get more users, to my product/service. And the problem is to get the presence in front of developers in order to get to know my app and use for their development. In current scenario, all the people got used to search  the information on Google, Bing, Ask, etc. So, I have to get onto the listing on those as well. And I have to use some direct and
Read more

SSH and Internet connection sharing over crossover ethernet cable from OS X (Mac) to Raspberry PI (model B)

Image
These days I am testing a Raspberry PI (model B) with 3.5" TFT LCD screen.  Since I have another Rasp. PI (model B+) to be setup and to remember these steps I wrote this with the thinking that any other person can also use this info. But I have only USB mobile broadband stick (dongle) and crossover Ethernet cable. So, I must connect to Raspberry PI via SSH (VNC) and I need Internet connection to the Rasp. PI as well. Since I have no Internet connection to Rasp. PI initially I cannot install anything on Rasp. PI in order to make VNC, apt-get update and install other necessary software. So, Finally I got through and now I have VNC with shared Internet connection over Ethernet cable from OS X. This is what I wanted at the end. Rasp. PI VNC view with Internet Rasp. PI setup: Rasp. PI with crossover Ethernet cable Though I share Internet connection of mobile broadband to Wi-fi (bridge0 has en1 as a member), in this document I am not using shared Wi-fi connec
Read more