Stasher

Prototyping a logstash alternative
23 February 2017

Silvan Jegen
Computational Linguist, BioVariance GmbH
me@sillymon.ch
https://sillymon.ch


* Logstash

.image img/icon-logstash-bb.png
.caption logstash logo from the [[https://www.elastic.co/products/logstash][official logstash site]]


* What is it?

.image img/logstash-img1.png


* "Centralize, Transform & Stash"

- Input
- Filters
- Outputs

Examples

- Input:
- Filters:
- Outputs:


* How does it work?

Custom configuration language

	input { stdin {} }

	filter {
	  http {
		"A" => "url:port"
	  }
	}

	output {
	  elasticsearch { hosts => ["localhost:9200"] }
	  stdout { codec => rubydebug }
	}


* Modules

	output {
	  stdout { codec => rubydebug }
	}


* Some statistics

- Written in Ruby
- 10000 LOC in 999 files


* Stasher

Why?

- Apparently Logstash is very slow (see ElasticSearch Bumbles)
- I like Go
- Generality (error handling)
- Or not?


* Implementation

- Plugins?
- Generality and error handling


* Demo


* Todos

- Watch input directories
- Multiple modules for each main module
- Proper (configurable?) error handling
- DSL/declarative vs. Programming language balance?


* Considerations

- Plugins (Go 1.8!?)
- Better off with shell scripts?

Further Text, including invocations like:

.link http://foo label
.caption _Gopher_ by [[http://www.reneefrench.com][Renée French]]

Again, more text