Building a Server Monitor: Pyramid Setup

Jan. 13, 2018, 2:53 p.m. by Avery Uslaner

Tags: server python pyramid automation monitor

Building a Server Monitor

A few days ago I realized how nice it would be to have a web interface to manage my various servers. So today I'm going to start building one. I'll also be using this as an excuse to try out the Python web framework Pyramid for the first time. As usual, this post will be less of a guide, and more of a record of my fumbling through the void that is the internet, trying to make things do what I want.

First off, we'll create the pyramid project. I'm using PyCharm as my IDE, it actually handles a lot of the initial project setup as far as my environment goes.

According to the Pyramid docs, its helpful to start with a cookiecutter version of a project using a package of the same name, so I guess I'll obey. I happen to be on my Windows box while starting this project, so I'll be installing it using conda:

conda config --add channels conda-forge
conda install cookiecutter

Now we have a few options for which cookiecutter to use. The pyramid-cookiecutter-alchemy project template sounds good to me since it uses persistent storage controlled by SQLAlchemy which I like a lot.

cookiecutter gh:Pylons/pyramid-cookiecutter-alchemy

This command then asks what you want to call the project and repo. Annoyingly, I couldn't call both things server_monitor because of directory conflicts with either my virtual environment, or my PyCharm project folder. Whatever it was, I just had to settle calling my repo serv_mon instead.


Oh...

I now realize that I was essentially reimplementing the steps that PyCharm already took for me. So I just started a Pyramid project inside of a Pyramid project. Hurrumph.

I'm just going to wipe the premade PyCharm files so I can follow along with the Pyramid docs. The silver lining is that now I can just repeat the steps above and actually name my project and repo server_monitor!

Now I need to install the project into my virtual environment. For some reason I get some crazy nonsense when I try to install using PyCharm's terminal:

Downloading SQLAlchemy-1.2.0.tar.gz (5.5MB)
    33% |ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûè                     | 1.8MB 2.3MB/s eta 0:00:02Exception:
Traceback (most recent call last):
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\commands\install.py", line 335, in run
    wb.build(autobuilding=True)
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\req\req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\req\req_set.py", line 620, in _prepare_file
    session=self.session, hashes=hashes)
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\download.py", line 821, in unpack_url
    hashes=hashes
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\download.py", line 659, in unpack_http_url
    hashes)
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\download.py", line 882, in _download_http_url
    _download_url(resp, link, content_file, hashes)
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\download.py", line 603, in _download_url
    hashes.check_against_chunks(downloaded_chunks)
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\utils\hashes.py", line 46, in check_against_chunks
    for chunk in chunks:
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\download.py", line 571, in written_chunks
    for chunk in chunks:
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\utils\ui.py", line 141, in iter
    self.next(n)
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\_vendor\progress\__init__.py", line 73, in next
    self.update()
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\_vendor\progress\bar.py", line 79, in update
    self.writeln(line)
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\_vendor\progress\helpers.py", line 68, in writeln
    print(line, end='', file=self.file)
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\_vendor\colorama\ansitowin32.py", line 141, in write
    self.write_and_convert(text)
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\_vendor\colorama\ansitowin32.py", line 169, in write_and_convert
    self.write_plain_text(text, cursor, len(text))
  File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\_vendor\colorama\ansitowin32.py", line 175, in write_plain_text
    self.wrapped.flush()
OSError: raw write() returned invalid length 180 (should have been between 0 and 90)
    33% |ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûè                     | 1.8MB 2.3MB/s eta 0:00:02

I assume it has something to do with the reference to cursor, but I'm not actually going to bother trying to figure out for sure because the install works just fine in my normal command prompt:

cd project_dir
pip install -e .

Now we install the testing dependencies because we're good developers who test things:

pip install -e ".[testing]"

And now we can run our tests using:

pytest -q

Which runs the two sample tests that were created by our cookiecutter template.

We can also start our project using pserve and our development configuration:

pserve development.ini

This launches the development web server. But when I navigate over to it in my browser, I'm told I need to do more stuff first:

Pyramid is having a problem using your SQL database.  The problem
might be caused by one of the following things:

1.  You may need to run the "initialize_server_monitor_db" script
    to initialize your database tables.  Check your virtual
    environment's "bin" directory for this script and try to run it.

2.  Your database server may not be running.  Check that the
    database server referred to by the "sqlalchemy.url" setting in
    your "development.ini" file is running.

After you fix the problem, please restart the Pyramid application to
try it again.

On Windows, this is a little annoying because the database initialization executable is in the directory of my virtual environment but I need to pass it the development.ini file as an argument:

C:\Users\Smith\Anaconda2\envs\server_monitor\Scripts\initialize_server_monitor_db.exe development.ini

Cool, now we can actually start our server.

Pyramid Project Initial Landing PageNext, I'll actually write some code myself.