logstasher

Logstasher

Awesome Logging for Rails !!

This gem is heavily inspired from lograge, but it's focused on one thing and one thing only. That's making your logs awesome like this: How it's done ?

By, using these awesome tools:

• Logstash - Store and index your logs
• Kibana - for awesome visualization. This is optional though, and you can use any other visualizer

Update: Logstash now includes Kibana build in, so no need to separately install. Logstasher has been tested with logstash with file input and json codec.

See quickstart for quickly setting up logstash

About logstasher

This gem purely focuses on how to generate logstash compatible logs i.e. logstash json event format, without any overhead. Infact, logstasher logs to a separate log file named logstash_.log. The reason for this separation:

• To have a pure json log file
• Prevent any logger messages(e.g. info) getting into our pure json logs

Before logstasher :

…

After logstasher:

…

By default, the older format rails request logs are disabled, though you can enable them.

Installation

In your Gemfile:

gem 'logstasher'

Configure your .rb e.g. development.rb

…

Optionally use config/logstasher.yml (overrides .rb)

Has the same optional fields as the .rb. You can specify common configurations that are then overriden by environment specific configurations:

controller_enabled: true
mailer_enabled: false
record_enabled: false
job_enabled: false
view_enabled: true
suppress_app_log: false
development:
  enabled: true
  record_enabled: true
production:
  enabled: true
  mailer_enabled: true
  view_enabled: false

Logging params hash

Logstasher can be configured to log the contents of the params hash. When enabled, the contents of the params hash (minus the ActionController internal params) will be added to the log as a deep hash. This can cause conflicts within the Elasticsearch mappings though, so should be enabled with care. Conflicts will occur if different actions (or even different applications logging to the same Elasticsearch cluster) use the same params key, but with a different data type (e.g. a string vs. a hash). This can lead to lost log entries. Enabling this can also significantly increase the size of the Elasticsearch indexes.

To enable this, add the following to your .rb

# Enable logging of controller params
config.logstasher.log_controller_parameters = true

Adding custom fields to the log

Since some fields are very specific to your application for e.g. user_name, so it is left up to you, to add them. Here's how to add those fields to the logs:

…

ruby

In config/initializers/logstasher.rb

Watch calls the block with the same arguments than any ActiveSupport::Notification, plus a store

LogStasher.watch('some.activesupport.notification') do |name, start, finish, id, payload, store|

Do something

store[:count] = 42 end

Would change the log entry to:

```json
{"@source":"unknown","@tags":["request"],"@fields":{"method":"GET","path":"/","format":"html","controller":"file_servers","action":"index","status":200,"duration":28.34,"view":25.96,"db":0.88,"ip":"127.0.0.1","route":"file_servers#index", "parameters":"","ndapi_time":null,"uuid":"e81ecd178ed3b591099f4d489760dfb6","user":"[email protected]", "site":"internal","some.activesupport.notification":{"count":42}},"@timestamp":"2013-04-30T13:00:46.354500+00:00"}

The store exposed to the blocked passed to watch is thread-safe, and reset after each request. By default, the store is only shared between occurences of the same event. You can easily share the same store between different types of notifications, by assigning them to the same event group:

# In config/initializers/logstasher.rb

LogStasher.watch('foo.notification', event_group: 'notification') do |*args, store|
  # Shared store with 'bar.notification'
end

LogStasher.watch('bar.notification', event_group: 'notification') do |*args, store|
  # Shared store with 'foo.notification'
end

Quick Setup for Logstash

Follow the instructions at logstash documentation to setup logstash. Start logstash with the following command:

bin/logstash -f quickstart.conf

Versions

Latest version require Rails 7/8 and higher and Ruby 3.0+

Development

• Install dependencies: export RAILS_VERSION=5.2.0 bundle install --without guard --path=${BUNDLE_PATH:-vendor/bundle}
• Run tests - rake
• Generate test coverage report - rake coverage. Coverage report path - coverage/index.html

Copyright

Copyright (c) 2016 Shadab Ahmed, released under the MIT license