NATS in action
A Real time Microservices Architecture handled by NATS
1
Who am I?
Raül Pérez
Senior Engineer at Carrenza Ltd.
Twitter: @repejota
Github: https://github.com/repejota
Email: repejo...
Who am I?
Raül Pérez
Senior Engineer at Carrenza Ltd.
Twitter: @repejota
Github: https://github.com/repejota
Email: repejo...
Who am I?
Raül Pérez
Senior Engineer at Carrenza Ltd.
Twitter: @repejota
Github: https://github.com/repejota
Email: repejo...
Who am I?
Raül Pérez
Senior Engineer at Carrenza Ltd.
Twitter: @repejota
Github: https://github.com/repejota
Email: repejo...
Who am I?
Raül Pérez
Senior Engineer at Carrenza Ltd.
Twitter: @repejota
Github: https://github.com/repejota
Email: repejo...
Who am I?
Raül Pérez
Senior Engineer at Carrenza Ltd.
Twitter: @repejota
Github: https://github.com/repejota
Email: repejo...
Microservices
8
Microservices
● Small autonomous services that work together.
9
● Divide and conquer approach.
Microservices
● Small autonomous services that work together.
● Communicate with lightweight mechanisms.
10
● Divide and c...
Microservices
● Small autonomous services that work together.
● Communicate with lightweight mechanisms.
11
● Divide and c...
Microservices
● Small autonomous services that work together.
● Communicate with lightweight mechanisms.
● The platform is...
Microservices
● Small autonomous services that work together.
● Communicate with lightweight mechanisms.
● The platform is...
Microservices … Why?
● It is quick to (re) write and to deploy.
14
● Simple application can be handled by less developers.
Microservices … Why?
● It is quick to (re) write and to deploy.
15
● Simple application can be handled by less developers....
Microservices … Why?
● It is quick to (re) write and to deploy.
● It knows that failures exist and how to handle them.
16
...
Microservices … Why?
● It is quick to (re) write and to deploy.
● It knows that failures exist and how to handle them.
● K...
Microservices … Why?
● It is quick to (re) write and to deploy.
● It knows that failures exist and how to handle them.
● K...
NATS
19
What is NATS?
● Cloud-native message platform.
20
● Designed to natively support modern cloud architectures.
What is NATS?
● Cloud-native message platform.
● Highly performant.
21
● Designed to natively support modern cloud archite...
What is NATS?
● Cloud-native message platform.
● Highly performant.
● Extremely lightweight.
22
● Designed to natively sup...
What is NATS?
● Cloud-native message platform.
● Highly performant.
● Extremely lightweight.
● Support for various messagi...
What is NATS?
● Cloud-native message platform.
● Highly performant.
● Extremely lightweight.
● Support for various messagi...
Ernest
IAAS+PAAS Hybrid Cloud
Platform
25
Ernest : IAAS+PAAS Hybrid Cloud Platform
● Manage your infrastructure. (IAAS)
26
● Manage servers, networks, balancers etc...
Ernest : IAAS+PAAS Hybrid Cloud Platform
● Manage your infrastructure. (IAAS)
● Manage your platform. (PAAS)
27
● Manage s...
Ernest : IAAS+PAAS Hybrid Cloud Platform
● Manage your infrastructure. (IAAS)
● Manage your platform. (PAAS)
● Scaling.
28...
Ernest : IAAS+PAAS Hybrid Cloud Platform
● Manage your infrastructure. (IAAS)
● Manage your platform. (PAAS)
● Scaling.
29...
Ernest : IAAS+PAAS Hybrid Cloud Platform
● Manage your infrastructure. (IAAS)
● Manage your platform. (PAAS)
● Scaling.
● ...
Architecture of Ernest
31
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
Public
API
&
UI
Monitor
Provider
Architecture of Ernest
32
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
Public
API
&
UI
Monitor
Provider
REST API
Architecture of Ernest
33
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
Public
API
&
UI
Monitor
Provider
REST AP...
Architecture of Ernest
34
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
Public
API
&
UI
Monitor
Provider
REST AP...
Architecture of Ernest
35
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
Public
API
&
UI
Monitor
Provider
REST AP...
Architecture of Ernest
36
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
Public
API
&
UI
Monitor
Provider
REST AP...
Architecture of Ernest
37
Provider
Provider
Provider
MicroservicesNATS
FSM
LogsDB
Public
API
&
UI
Monitor
Provider
REST AP...
Ernest + NATS
38
Ernest + NATS
● NATS is the central communication system for Ernest.
39
● Ernest core is a series of microservices talking...
Ernest + NATS
● NATS is the central communication system for Ernest.
● Ernest executes workflows defined by the user to bu...
Workflow example
41
---
name: my-awesome-example
networks:
- name: network-web
subnet: 10.2.0.0/24
instances:
- name: inst...
Workflow example
42
---
name: my-awesome-example
networks:
- name: network-web
subnet: 10.2.0.0/24
instances:
- name: inst...
Workflow example
43
---
name: my-awesome-example
networks:
- name: network-web
subnet: 10.2.0.0/24
instances:
- name: inst...
Workflow example
44
---
name: my-awesome-example
networks:
- name: network-web
subnet: 10.2.0.0/24
instances:
- name: inst...
Workflow example
45
---
name: my-awesome-example
networks:
- name: network-web
subnet: 10.2.0.0/24
instances:
- name: inst...
Workflow example
46
---
name: my-awesome-example
networks:
- name: network-web
subnet: 10.2.0.0/24
instances:
- name: inst...
Ernest + NATS
● NATS is the central communication system for Ernest.
● A workflow composed by multiple actions is executed...
Ernest + NATS
● NATS is the central communication system for Ernest.
● A workflow composed by multiple actions is executed...
Ernest : Numbers
● Less than 5 developers.
49
● A developer “owns” a component / microservice.
Ernest : Numbers
● Less than 5 developers.
● Started as a prototype in Ruby, moving almost everything to Go.
50
● A develo...
Ernest : Numbers
● Less than 5 developers.
● Started as a prototype in Ruby, moving almost everything to Go.
● Almost 50 m...
Ernest : Numbers
● Less than 5 developers.
● Started as a prototype in Ruby, moving almost everything to Go.
● Almost 50 m...
Ernest : Numbers
● Less than 5 developers.
● Started as a prototype in Ruby, moving almost everything to Go.
● Almost 50 m...
Resources
● http://microservices.io
● https://en.wikipedia.org/wiki/Cloud_computing
● http://nats.io
54
● http://carrenza....
Questions?
55
Thank you!
56
of 56

Nats in action a real time microservices architecture handled by nats

A talk about NATS ( http://nats.io ) and how we use this cloud message system at Carrenza Ltd. in our product called Ernest. London - 20/10/2015
Published on: Mar 3, 2016
Published in: Software      
Source: www.slideshare.net


Transcripts - Nats in action a real time microservices architecture handled by nats

  • 1. NATS in action A Real time Microservices Architecture handled by NATS 1
  • 2. Who am I? Raül Pérez Senior Engineer at Carrenza Ltd. Twitter: @repejota Github: https://github.com/repejota Email: repejota@gmail.com 2
  • 3. Who am I? Raül Pérez Senior Engineer at Carrenza Ltd. Twitter: @repejota Github: https://github.com/repejota Email: repejota@gmail.com 3 ● Almost 4 years working on devop & distributed projects.
  • 4. Who am I? Raül Pérez Senior Engineer at Carrenza Ltd. Twitter: @repejota Github: https://github.com/repejota Email: repejota@gmail.com 4 ● Almost 4 years working on devop & distributed projects. ● Still more a dev than op dude.
  • 5. Who am I? Raül Pérez Senior Engineer at Carrenza Ltd. Twitter: @repejota Github: https://github.com/repejota Email: repejota@gmail.com 5 ● Almost 4 years working on devop & distributed projects. ● Still more a dev than op dude. ● Proud to be a Gopher after a looong PHP/Ruby/Python/Node.js past experience. :P
  • 6. Who am I? Raül Pérez Senior Engineer at Carrenza Ltd. Twitter: @repejota Github: https://github.com/repejota Email: repejota@gmail.com 6 ● Almost 4 years working on devop & distributed projects. ● Still more a dev than op dude. ● Proud to be a Gopher after a looong PHP/Ruby/Python/Node.js past experience. :P ● Based in Barcelona.
  • 7. Who am I? Raül Pérez Senior Engineer at Carrenza Ltd. Twitter: @repejota Github: https://github.com/repejota Email: repejota@gmail.com 7 ● Almost 4 years working on devop & distributed projects. ● Still more a dev than op dude. ● Proud to be a Gopher after a looong PHP/Ruby/Python/Node.js past experience. :P ● Based in Barcelona. ● Love startups & love remote work!
  • 8. Microservices 8
  • 9. Microservices ● Small autonomous services that work together. 9 ● Divide and conquer approach.
  • 10. Microservices ● Small autonomous services that work together. ● Communicate with lightweight mechanisms. 10 ● Divide and conquer approach. ● HTTP/REST microservices. ( Request / Response )
  • 11. Microservices ● Small autonomous services that work together. ● Communicate with lightweight mechanisms. 11 ● Divide and conquer approach. ● HTTP/REST microservices. ( Request / Response ) ● Message Systems for asynchronous communication. ( Publish/Subscribe )
  • 12. Microservices ● Small autonomous services that work together. ● Communicate with lightweight mechanisms. ● The platform is composed of one or more stateless processes. 12 ● Divide and conquer approach. ● HTTP/REST microservices. ( Request / Response ) ● Message Systems for asynchronous communication. ( Publish/Subscribe ) ● There is no shared data between different processes.
  • 13. Microservices ● Small autonomous services that work together. ● Communicate with lightweight mechanisms. ● The platform is composed of one or more stateless processes. ● Designed to be distributed. 13 ● Divide and conquer approach. ● HTTP/REST microservices. ( Request / Response ) ● Message Systems for asynchronous communication. ( Publish/Subscribe ) ● There is no shared data between different processes. ● Scaling is easy, each microservice can be executed in a different server.
  • 14. Microservices … Why? ● It is quick to (re) write and to deploy. 14 ● Simple application can be handled by less developers.
  • 15. Microservices … Why? ● It is quick to (re) write and to deploy. 15 ● Simple application can be handled by less developers. ● Developers do not need to know the entire platform to be productive.
  • 16. Microservices … Why? ● It is quick to (re) write and to deploy. ● It knows that failures exist and how to handle them. 16 ● Simple application can be handled by less developers. ● Developer do not need to know the entire platform to be productive. ● Defensive programming, services should not break.
  • 17. Microservices … Why? ● It is quick to (re) write and to deploy. ● It knows that failures exist and how to handle them. ● KISS: Keep It Simple and Stupid. 17 ● Simple application can be handled by less developers. ● Developer do not need to know the entire platform to be productive. ● Defensive programming, services should not break. ● It must provide one functionality.
  • 18. Microservices … Why? ● It is quick to (re) write and to deploy. ● It knows that failures exist and how to handle them. ● KISS: Keep It Simple and Stupid. ● It must provide an API that other services can use. 18 ● Simple application can be handled by less developers. ● Developer do not need to know the entire platform to be productive. ● Defensive programming, services should not break. ● It must provide one functionality. ● Rely on best practices.
  • 19. NATS 19
  • 20. What is NATS? ● Cloud-native message platform. 20 ● Designed to natively support modern cloud architectures.
  • 21. What is NATS? ● Cloud-native message platform. ● Highly performant. 21 ● Designed to natively support modern cloud architectures. ● It is fast! Can handle millions of messages per second.
  • 22. What is NATS? ● Cloud-native message platform. ● Highly performant. ● Extremely lightweight. 22 ● Designed to natively support modern cloud architectures. ● It is fast! Can handle millions of messages per second. ● A single binary, easy to deploy without unnecessary complexity.
  • 23. What is NATS? ● Cloud-native message platform. ● Highly performant. ● Extremely lightweight. ● Support for various messaging models and use cases. 23 ● Designed to natively support modern cloud architectures. ● It is fast! Can handle millions of messages per second. ● A single binary, easy to deploy without unnecessary complexity. ● Request/Response, Publish/Subscribe, Queues.
  • 24. What is NATS? ● Cloud-native message platform. ● Highly performant. ● Extremely lightweight. ● Support for various messaging models and use cases. 24 ● Designed to natively support modern cloud architectures. ● It is fast! Can handle millions of messages per second. ● A single binary, easy to deploy without unnecessary complexity. ● Request/Response, Publish/Subscribe, Queues. ● Simple text based protocol.
  • 25. Ernest IAAS+PAAS Hybrid Cloud Platform 25
  • 26. Ernest : IAAS+PAAS Hybrid Cloud Platform ● Manage your infrastructure. (IAAS) 26 ● Manage servers, networks, balancers etc ... of your platform.
  • 27. Ernest : IAAS+PAAS Hybrid Cloud Platform ● Manage your infrastructure. (IAAS) ● Manage your platform. (PAAS) 27 ● Manage servers, networks, balancers etc ... of your platform. ● Deploy and upgrade your application to the desired servers.
  • 28. Ernest : IAAS+PAAS Hybrid Cloud Platform ● Manage your infrastructure. (IAAS) ● Manage your platform. (PAAS) ● Scaling. 28 ● Manage servers, networks, balancers etc ... of your platform. ● Deploy and upgrade your application to the desired servers. ● Automate everything.
  • 29. Ernest : IAAS+PAAS Hybrid Cloud Platform ● Manage your infrastructure. (IAAS) ● Manage your platform. (PAAS) ● Scaling. 29 ● Manage servers, networks, balancers etc ... of your platform. ● Deploy and upgrade your application to the desired servers. ● Automate everything. ● Evolve your platform when you need and redeploy your platform.
  • 30. Ernest : IAAS+PAAS Hybrid Cloud Platform ● Manage your infrastructure. (IAAS) ● Manage your platform. (PAAS) ● Scaling. ● Hybrid (Multiple Providers) 30 ● Manage servers, networks, balancers etc ... of your platform. ● Deploy and upgrade your application to the desired servers. ● Evolve your infrastructure when you need and redeploy your platform. ● Automate everything. ● Do not rely on a single provider, use different ones with a single interface.
  • 31. Architecture of Ernest 31 Provider Provider Provider MicroservicesNATS FSM LogsDB Public API & UI Monitor Provider
  • 32. Architecture of Ernest 32 Provider Provider Provider MicroservicesNATS FSM LogsDB Public API & UI Monitor Provider REST API
  • 33. Architecture of Ernest 33 Provider Provider Provider MicroservicesNATS FSM LogsDB Public API & UI Monitor Provider REST API Events + State
  • 34. Architecture of Ernest 34 Provider Provider Provider MicroservicesNATS FSM LogsDB Public API & UI Monitor Provider REST API Events + State Business Logic Real Time(almost)
  • 35. Architecture of Ernest 35 Provider Provider Provider MicroservicesNATS FSM LogsDB Public API & UI Monitor Provider REST API Events + State Business Logic Real Time(almost) VMWare
  • 36. Architecture of Ernest 36 Provider Provider Provider MicroservicesNATS FSM LogsDB Public API & UI Monitor Provider REST API Events + State Business Logic Real Time(almost) VMWare AWS
  • 37. Architecture of Ernest 37 Provider Provider Provider MicroservicesNATS FSM LogsDB Public API & UI Monitor Provider REST API Events + State Business Logic Real Time(almost) VMWare AWS Others...
  • 38. Ernest + NATS 38
  • 39. Ernest + NATS ● NATS is the central communication system for Ernest. 39 ● Ernest core is a series of microservices talking each other through NATS.
  • 40. Ernest + NATS ● NATS is the central communication system for Ernest. ● Ernest executes workflows defined by the user to build environments. 40 ● Ernest core is a series of microservices talking each other through NATS. ● Users posts a workflow written in a YAML file to be processed.
  • 41. Workflow example 41 --- name: my-awesome-example networks: - name: network-web subnet: 10.2.0.0/24 instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run ● Simple example.
  • 42. Workflow example 42 --- name: my-awesome-example networks: - name: network-web subnet: 10.2.0.0/24 instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run ● Simple example. ● Each environment has an unique name.
  • 43. Workflow example 43 --- name: my-awesome-example networks: - name: network-web subnet: 10.2.0.0/24 instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run ● Simple example. ● Each environment has an unique name. ● Action: Create a network on our provider.
  • 44. Workflow example 44 --- name: my-awesome-example networks: - name: network-web subnet: 10.2.0.0/24 instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run ● Simple example. ● Each environment has an unique name. ● Action: Create a network on our provider. ● Action: Create an instance type for our app.
  • 45. Workflow example 45 --- name: my-awesome-example networks: - name: network-web subnet: 10.2.0.0/24 instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run ● Simple example. ● Each environment has an unique name. ● Action: Create a network on our provider. ● Action: Create an instance type for our app. ● Count determines the number of servers.
  • 46. Workflow example 46 --- name: my-awesome-example networks: - name: network-web subnet: 10.2.0.0/24 instances: - name: instance-web image: images/centos65-tty-sudo-disabled-nodejs-mongodb count: 1 cpus: 1 memory: 4GB disks: - 40GB network: name: network-web start_ip: "10.2.0.100" provisioner: - git clone https://host/user/myapp && make install && make run ● Simple example. ● Each environment has an unique name. ● Action: Create a network on our provider. ● Action: Create an instance type for our app. ● Count determines the number of servers. ● Action: Deploy your application and run it.
  • 47. Ernest + NATS ● NATS is the central communication system for Ernest. ● A workflow composed by multiple actions is executed by the user. ● A finite state machine handles the state for the platform. 47 ● Ernest core is a series of microservices talking each other through NATS. ● Users posts a workflow written in a YAML file to be processed. ● A list of actions are executed, one per microservice.
  • 48. Ernest + NATS ● NATS is the central communication system for Ernest. ● A workflow composed by multiple actions is executed by the user. ● A finite state machine handles the state for the platform. ● Publish Subscribe is the main pattern used. 48 ● Ernest core is a series of microservices talking each other through NATS. ● Users posts a workflow written in a YAML file to be processed. ● A list of actions are executed, one per microservice. ● Actions are published to NATS ( by a topic ) and microservices are subscribed to execute them.
  • 49. Ernest : Numbers ● Less than 5 developers. 49 ● A developer “owns” a component / microservice.
  • 50. Ernest : Numbers ● Less than 5 developers. ● Started as a prototype in Ruby, moving almost everything to Go. 50 ● A developer “owns” a component / microservice. ● Fast to develop, less errors, less dependencies.
  • 51. Ernest : Numbers ● Less than 5 developers. ● Started as a prototype in Ruby, moving almost everything to Go. ● Almost 50 microservices. 51 ● A developer “owns” a component / microservice. ● Fast to develop, less errors, less dependencies. ● Unix philosophy, do one thing and do it well done.
  • 52. Ernest : Numbers ● Less than 5 developers. ● Started as a prototype in Ruby, moving almost everything to Go. ● Almost 50 microservices. ● Everything specified. Everything tested. 52 ● A developer “owns” a component / microservice. ● Fast to develop, less errors, less dependencies. ● Unix philosophy, do one thing and do it well done. ● Using TDD on each part of the platform.
  • 53. Ernest : Numbers ● Less than 5 developers. ● Started as a prototype in Ruby, moving almost everything to Go. ● Almost 50 microservices. ● Everything specified. Everything tested. 53 ● A developer “owns” a component / microservice. ● Fast to develop, less errors, less dependencies. ● Unix philosophy, do one thing and do it well done. ● Using TDD on each part of the platform. ● Documentation on APIs, documentation on payloads.
  • 54. Resources ● http://microservices.io ● https://en.wikipedia.org/wiki/Cloud_computing ● http://nats.io 54 ● http://carrenza.com ● http://apcera.com
  • 55. Questions? 55
  • 56. Thank you! 56

Related Documents