用于 ML 项目的日志记录器、服务器和可视化仪表板
ML-Logger makes it easy to:
pickle file, with the same API and zero
configuration.matplotlib.pyplot figures from a remote server locally with logger.savefig('my_figure.png')And ml-logger does all of these with minimal configuration — you can use the same logging code both locally and remotely with no code change.
ML-logger is highly performant -- the remote writes are asynchronous. For this reason it doesn't slow down your training even with 100+ metric keys.
Why did we built this, you might ask? Because we want to make it easy for people in ML to use the same logging code in all of they projects, so that it is easy to get started with someone else's baseline.
To install ml_logger, do:
pip install ml-loggerThe landscape of python modules is a lot messier than that of javascript. The most up-to-date
graphenerequires the following versions:pip install sanic==20.9.0 pip install sanic-cors==0.10.0.post3 pip install sanic-graphql==1.1.0 yes | pip install graphene==2.1.3 yes | pip install graphql-core==2.1 yes | pip install graphql-relay==0.4.5 yes | pip install graphql-server-core==1.1.1
Add the following to your ~/.zshrc
export ML_LOGGER_ROOT=
export ML_LOGGER_USER= # make sure it is the same on the cluster.Now you can rock!
from ml_logger import logger
print(logger)
# ~> logging data to /tmp/ml-logger-debugLog key/value pairs, and metrics:
for i in range(1):
logger.log(metrics={'some_val/smooth': 10, 'status': f"step ({i})"}, reward=20, timestep=i)
### flush the data, otherwise the value would be overwritten with new values in the next iteration.
logger.flush()
# outputs ~>
# ╒════════════════════╤════════════════════════════╕
# │ reward │ 20 │
# ├────────────────────┼────────────────────────────┤
...In your project files, do:
…And it should print out
…Note, the jwt based access control has not been implemented yet. So you should leave the token field empty.
Skip this if you just want to log locally. When training in parallel, you want to kickstart an logging server (Instrument Server). To do so, run:
python -m ml_logger.serverUse ssh tunnel if you are running on a managed cluster (with SLURM for instance).
Important: to set allow remote logging, you need to pass in --host=0.0.0.0 so that the server accepts non-localhost connections.
python -m ml_logger.server --host=0.0.0.0A common scenario is you only want to upload averaged statistics of your metrics. A pattern
that @jachiam uses is the following: store_metrics(), peak_stored_metrics(), and log_metrics_summary()
…matplotlib.pyplot figures on an headless serverFirst clone repo, install dev dependencies, and install the module under evaluation mode.
git clone https://github.com/episodeyang/ml_logger.git
cd ml_logger && cd ml_logger && pip install -r requirements-dev.txt
pip install -e .You should be inside ml_logger/ml_logger folder
pwd # ~> ml_logger/ml_logger
make testTo test with a live server, first run (in a separate console)
python -m ml_logger.server --data-dir /tmp/ml-logger-debugor do:
make start-test-serverThen run this test script with the option:
python -m pytest tests --capture=no --data-dir http://0.0.0.0:8081or do
make test-with-serverYour PR should have both of these two tests working. ToDo: add CI to this repo.
You need twine, rst-lint etc, which are included in the requirements-dev.txt file.
from ml_logger import logger, ML_Logger
from datetime import datetime
now = datetime.now()
logger.configure("/tmp/ml-logger-demo", "deep_Q_learning", f"{now:%Y%m%d-%H%M%S}")This is a singleton pattern similar to matplotlib.pyplot. However, you could also use the logger constructor
logger = ML_Logger(root_dir="/tmp/ml-logger-demo", prefix=f"deep_Q_learning/{now:%Y%m%d-%H%M%S}")logger.log({"some_var/smooth": 10}, some=Color(0.85, 'yellow', percent), step=3)colored output: (where the values are yellow)
╒════════════════════╤════════════════════╕
│ some var/smooth │ 10 │
├────────────────────┼────────────────────┤
│ some │ 85.0% │
╘════════════════════╧════════════════════╛We have optimized ML-Logger, so it supports any format that pyplot supports. To save a figure locally or remotely,
import matplotlib.pyplot as plt
import numpy as np
xs = np.linspace(-5, 5)
plt.plot(xs, np.cos(xs), label='Cosine Func')
logger.savefig('cosine_function.pdf')It is especially hard to visualize RL training sessions on a remote computer. With ML-Logger this is easy, and
super fast. We optimized the serialization and transport process, so that a large stack of video tensor gets
first compressed by ffmepg before getting sent over the wire.
The compression rate (and speed boost) can be 2000:1.
import numpy as np
def im(x, y):
canvas = np.zeros((200, 200))
for i in range(200):
for j in range(200):
if x - 5 )` is an http end point,
the logger will instantiate a fast, future based logging client that launches
http requests in a separate thread. We optimized the client so that it won't
slow down your training code.
API wise, ML-logger makes it easy for you to log textual printouts, simple
scalars, numpy tensors, image tensors, and `pyplot` figures. Because you might
also want to read data from the instrumentation server, we also made it possible to
load numpy, pickle, text and binary files remotely.
In the future, we will start building an integrated dashboard with fast search,
live figure update and markdown-based reporting/dashboarding to go with ml-logger.
Now give this a try, and profit!暂无开放 Issues,或尚未同步最近议题。