Repeating History...On Purpose...with Elixir
-
Upload
barry-jones -
Category
Technology
-
view
1.104 -
download
1
Transcript of Repeating History...On Purpose...with Elixir
Repeating History…on purpose…
with Elixir
Barry Jones
Who am I?
• Developer since 98• Used to run Brightball– Contract programming business using PHP– 2008 – 2011• R.I.P. Code one out for my homies
• PHP, Java, Perl, Python, Ruby, Groovy, Go– Elixir
• And lots of databases #postgresql
So…another language…? Rules for learning a language– Must solve a problem– Problem not addressed by
current stack– “Fast” is not a purpose• Lots of things are fast
– Should be the best solution• If it’s not, I’d rather learn the
one that is…
I hate new languages
Yeah. Surprise.
So why the other languages?• PHP
– Needed to build a site– Clemson’s options were PHP / Perl– Perl was badly supported
• Java– Clemson CPSC– Also…jobs are good
• Groovy– Java environment– Wanted to be productive
• Go– Great concurrency– Relatively simple– Fast compilation– Portability– Solves some bloat problems
• Perl– Sysadmin work– Installed on all *nix derivatives
• including very old– Best at text parsing
• Python– Employer required it
• Ruby– Incredible productivity– Library ecosystem– Monkey patching– Rails– Focus on dev efficiency
SO WHY ELIXIR?I wondered the same thing
"Those who cannot remember the past are condemned to repeat it."
- George Santayana
"What has been is what will be, and what has been done is what will be done, and there is
nothing new under the sun." - Solomon
Elixir isn’t new
• It’s more productive Erlang• Compiles down to the BEAM (Erlang VM)• Erlang was born in 1986 (Linux was 1991)• Erlang/OTP is on version 19• Erlang libraries work directly in Elixir– And vice versa
Processor POWER!!!
• Industry focus used to be on Mhz/Ghz• Growth was steady…but then stalled
• Industry shifted to multicore / concurrency and trying to put it in their languages
• Lookup “Beowulf Cluster” to see how foreign “parallel” was at the time
Funny thing…
“Any sufficiently complicated concurrent program in another language contains an ad hoc informally-specified bug-ridden slow implementation of half of Erlang.”- Robert Virding
Erlang view of the World• Everything is a process.• Processes are strongly isolated.• Process creation and destruction is a lightweight operation.• Message passing is the only way for processes to interact.• Processes have unique names.• If you know the name of a process you can send it a
message.• Processes share no resources.• Error handling is non-local.• Processes do what they are supposed to do or fail.
Message Passing
• Process isolation and message passing• Passing between threads, cores or machines is
transparent• Microservice concept…but everywhere and
not terrible
“If Java is the right one to run anywhere, then Erlang is the right one to run forever.” – Joe Armstrong
What’s the big deal with Elixir?• Ruby-like focus on developer productivity• Functional programming made simple• Embedded database• Compiles down to code to run on the BEAM Virtual Machine• BEAM/OTP is what Erlang runs on• Erlang/BEAM is the best existing language for concurrency, consistency and fault
tolerance, hot code swapping– Single Processor– Multi Processor– Distributed Multi Machine Cluster
• Erlang does not focus on developer productivity– Writing Erlang kinda sucks...
• #1 Problem in Ruby is concurrency model– Cannot be fixed. It’s the way the language works. – Enables great things but also causes limitations
So what does that mean?
Standard Web App OTP
What’s the big deal?
• Facebook paid $22 billion for WhatsApp• WhatsApp had $10 million in revenue• What was the big deal?– Erlang/OTP– 2 million users / server– No central relay point– Scales horizontally– Deploys w/o disconnect
AND HOW DOES ALL THIS WORK?Why should you care?
Other languages
• Boot up• Memory is shared– Where leaks come from– Changing shared memory requires a mutex lock
• Garbage collector periodically runs– Pause entire stack
• Requests run in threads in the same process– Threads are cooperatively scheduled
• Deployment means shutting down current code, starting new code
Erlang/Elixir/OTP
• No memory is shared• Data structures are immutable• Each Erlang process (basically a light thread) has
its own HEAP– Reclaimed on completion
• Code can be hot deployed– New code runs next time it’s accessed (existing code
keeps running)• Processes are prescheduled
Sound familiar?
• Difference is size of the allocations– An Erlang process is 0.5 kb– A Go goroutine is 2 kb (version 1.4)– A Java thread is 1024 kb on 64 bit VM– PHP request varies by how much is loaded• Laravel averages 7-12mb / request
Programming Elixir, Chapter 15Laptop w/ 4 cores and 4gb of RAM counting concurrently1,000,000 processes = • 0.48 gb in Elixir• 1.91 gb in Golang (go routines)• 977 gb in Java (threads)• 6,836 gb in PHP (Laravel requests)
LET’S GET STARTED
wait.…we haven’t started yet?
Quick History• Linux was created in 1991
• Erlang was created in 1987 by Ericson– Powers about half of global telecom– Needed distributed, fault tolerant system– Deploy updates without interrupting existing calls– OTP = Open Telecom Protocol– Erlang/OTP 19.0 release in June 2016
• Elixir was created in 2012 by Jose Valim– Former Rails Core team member– Elixir 1.3 released in June 2016
The HighlightsIt’s huge…we don’t have all night
Immutable Data
• There’s no passing pointers• Add something to a list, get a new list• Everything is “message passing”– Avoids mutex locks– Enables per-process garbage collection– Makes calling a function locally, in another process
or on another machine transparent
3 Databases Built In• ETS – Erlang Term Storage
– In memory table storage for a node
• DETS – Disk-based Erlang Term Storage– Disk table storage for a node
• Mnesia - #awesome– A relational/object hybrid data model that is suitable for telecommunications applications. – A DBMS query language, Query List Comprehension (QLC) as an add-on library. – Persistence. Tables can be coherently kept on disc and in the main memory. – Replication. Tables can be replicated at several nodes. – Atomic transactions. A series of table manipulation operations can be grouped into a single
atomic transaction. – Location transparency. Programs can be written without knowledge of the actual data location. – Extremely fast real-time data searches. – Schema manipulation routines. The DBMS can be reconfigured at runtime without stopping the
system.
https://blog.codeship.com/elixir-ets-vs-redis/
Preemptive Scheduling
• Context switching among running tasks and has the power to preempt (interrupt) tasks and resume them at a later time without the cooperation of the preempted tasks.
• Cooperative: Running tasks voluntarily release control
What does that mean?
• Response time consistency• A tight loop or resource heavy process can’t
cannibalize resources• Critical for real time systems• Running a database inside your code would be
unreliable otherwise
Pattern Matching
=iex> list = [1, 2, [ 3, 4, 5 ] ] [1, 2, [3, 4, 5]]iex> [a, b, c ] = list[1, 2, [3, 4, 5]] iex> a 1iex> b 2 iex> c [3, 4, 5]
Examples from Programming Elixir 1.3
Works if left can be matched to rightiex> list = [1, 2, 3][1, 2, 3]
iex> [a, 1, b ] = list** (MatchError) no match of right hand side value: [1, 2, 3]
Pattern Matching Functionsdefmodule Factorial do def of(0), do: 1 def of(n), do: n * of(n-1) end
defmodule PrintStuff do def print({:error, stuff}) do IO.puts “ERROR! #{stuff}” end def print({:ok, stuff}), do: IO.puts stuffend PrintStuff.print({:ok, stuff})
Loops?
• How do you have a for loop with an immutable increment?– Recursion. Lots of recursion.
Stack Overflow
Who knows what a stack overflow is?
Tail Call Optimizationdefmodule TailRecursive do def factorial(n), do: _fact(n, 1) defp _fact(0, acc), do: acc defp _fact(n, acc), do: _fact(n-1, acc*n)
end
# defp is private
If the last function called is itself, the stack doesn’t grow.
Concurrencypid = spawn(Object, :method, [vars])# Creates a process# returns the ID of the process
pid = spawn_link(Object, :method, [vars])# Creates a process# returns the ID of the process# If the process dies, creator should too
Quick Example (from book)defmodule Link2 do import :timer, only: [ sleep: 1 ] def sad_function do sleep 500 exit(:boom) end
def run do spawn_link(Link2, :sad_function, []) receive do msg -> IO.puts "MESSAGE RECEIVED: #{inspect msg}" after 1000 -> IO.puts "Nothing happened as far as I am concerned" end end End
Link2.run # The runtime reports the abnormal termination: $ elixir -r link2.exs** (EXIT from #PID<0.35.0>) :boom
That’s where we start
• Building blocks for best practice patterns– GenServer– Task (async/await)– Agent (async / await + state)– Supervisor / Worker
Fault Tolerance / Supervisors
• Applications operate as a Supervisor tree• Process is created with another process
dedicated to monitoring it• Worker process dies, it’s immediately
restarted in original state– This is how Erlang applications can get
99.9999999% uptime (yes, 9 nines)
HANDLING ERRORS IS CODE SMELLThis is just fun to say
Error that could kill process?
• If you have an error that could kill a process…– Make sure the process knows how to restart in a
desirable state
• Very different way of thinking about problems
Simple Supervision Example> Math.Calculate.divide(10,2)5.0:ok
> Math.Calculate.divide(34,3)11.333333333333334:ok
> Math.Calculate.divide(34,0)A BIG UGLY ERROR MESSAGE... BUT LITTLE DID YOU KNOW THE PROCESS WAS RESTARTED AND LIVES!
> Math.Calculate.divide(34,2)17.0:OK
https://github.com/kblake/simple-supervision
“Where you’d previously think Object you’ll begin to think Process“
- Confucius
Dialyzer
• Operators are not overridden• + is always math– What’s on either side of it is always a number
• Allows dynamic typing WITH compiler checks• Best of both worlds, problems of neither
THIS ALL SEEMS REALLY COMPLICATED
I know right? I’m just here for the web stuff
Phoenix Framework
• Significantly lowers barrier to entry– Familiar Routes / Controllers– Flexible Middleware
• Plug– Excellent web sockets
• Channels– Excellent / flexible / replaceable database layer
• Ecto– Best view layer ever– Pretty dang fast
About that….
Rebuilt brightball.com
• Read about ithttp://www.brightball.com/articles/insanity-with-elixir-phoenix-postgresql
• Summary– As a dynamic site it’s as fast as my static nginx site– This is considered normal (and awesome)
Phoenix Channels vs Rails ActionCable
• https://dockyard.com/blog/2016/08/09/phoenix-channels-vs-rails-action-cable
• Results– Rails: 50 rooms, 2500 users - .05s avg– Rails: 75 rooms, 3750 users – 8s avg, degrading– Phoenix: 1100 rooms, 55,000 users - .25s avg• (maxed 55,000 client connections)
Terraform
• Use in your current environment incrementally– Put Phoenix in front of your app– Route specific requests to Phoenix– Pass the rest through to your current app– Bundled Cowboy webserver used by Heroku, AWS
Cloudfront, Incapsula, etc…it good• https://medium.com/@sugarpirate/rise-from-t
he-ashes-incremental-apis-with-phoenix-b08cd66bd142#.e1iojykq9
GenStage
• http://elixir-lang.org/blog/2016/07/14/announcing-genstage/
• Streaming, auto-scaling data ingestion
Nerves
• http://nerves-project.org/• ElixirConf was about Phoenix and Nerves• Nerves is for embedded software• Fault tolerance, reliability, consistency seem
important for something like that…?
Functional Programming
• What are the gains?– Readability / Maintainability– No side effects code– Simplified testing– Simplified personnel turnover– No object inheritance nightmares– Clear separation of concerns
Objects…so dumb • Data tied to Functions– Breeds repetition– Weird nesting– Breeds repetition– Modify up the tree– Breeds repetition
• Separate the two– Data structures– Functions– #mindblown
"Object oriented programs are offered as alternatives to correct ones” - Edsger W. Dijkstra
http://www.yegor256.com/2016/08/15/what-is-wrong-object-oriented-programming.html
Why is it the future?• Nothing with a shared memory model can match it for distribution…ever
– You can’t pass a memory reference to another machine transparently• Ruby like productivity
– Minus an eventual full rewrite expectation• Extremely fast, small footprint, embeddable, consistent, reliable• Low process overhead ideal for holding connections
• You can easily start SMALL, knowing you have all of the tools to grow/get crazy when you need them– Refactoring is just rearranging stuff (yes, really)– Naturally avoids bloat
• Compiler tells you when stuff isn’t used to make cleanup easy• But isn’t so strict that it forces you to change them
– Stop worrying about development time OR code performance tradeoffs
Which makes it ideal for…
• Server applications that talk to a lot of things…• Like web sockets• Internet of Things devices• Real time communications• Low latency applications• Geographic distribution– Cluster across data centers…yes, really.
• Avoiding bottlenecks
Resources
• Hack Greenville Slack #elixir-phoenix• elixir-lang.org– Books, chat, links, resources, groups
• phoenixframework.org• Elixir Conf sessions on YouTube– So much good stuff– Using Phoenix w/ Riak Core https
://www.youtube.com/watch?v=sYYOLaJ-VDQ&start=2&autoplay=1
More Resources
• Programming Elixir• Programming Phoenix• RedFour (fake company training)• ElixirForums• Assorted newsletters
Elixir Tank Game
http://tanx.verse15.net/Go there now, fight to the death
THANKS!Questions?