Building a Server Monitor: Pyramid Setup
Jan. 13, 2018, 2:53 p.m. by Avery Uslaner
Building a Server Monitor
A few days ago I realized how nice it would be to have a web interface to manage my various servers. So today I'm going to start building one. I'll also be using this as an excuse to try out the Python web framework Pyramid for the first time. As usual, this post will be less of a guide, and more of a record of my fumbling through the void that is the internet, trying to make things do what I want.
First off, we'll create the pyramid project. I'm using PyCharm as my IDE, it actually handles a lot of the initial project setup as far as my environment goes.
According to the Pyramid docs, its helpful to start with a cookiecutter version of a project using a package of the same name, so I guess I'll obey. I happen to be on my Windows box while starting this project, so I'll be installing it using conda:
conda config --add channels conda-forge conda install cookiecutter
Now we have a few options for which cookiecutter to use. The pyramid-cookiecutter-alchemy project template sounds good to me since it uses persistent storage controlled by SQLAlchemy which I like a lot.
This command then asks what you want to call the project and repo. Annoyingly, I couldn't call both things server_monitor because of directory conflicts with either my virtual environment, or my PyCharm project folder. Whatever it was, I just had to settle calling my repo serv_mon instead.
I now realize that I was essentially reimplementing the steps that PyCharm already took for me. So I just started a Pyramid project inside of a Pyramid project. Hurrumph.
I'm just going to wipe the premade PyCharm files so I can follow along with the Pyramid docs. The silver lining is that now I can just repeat the steps above and actually name my project and repo server_monitor!
Now I need to install the project into my virtual environment. For some reason I get some crazy nonsense when I try to install using PyCharm's terminal:
Downloading SQLAlchemy-1.2.0.tar.gz (5.5MB) 33% |ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûè | 1.8MB 2.3MB/s eta 0:00:02Exception: Traceback (most recent call last): File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\basecommand.py", line 215, in main status = self.run(options, args) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\commands\install.py", line 335, in run wb.build(autobuilding=True) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\wheel.py", line 749, in build self.requirement_set.prepare_files(self.finder) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\req\req_set.py", line 380, in prepare_files ignore_dependencies=self.ignore_dependencies)) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\req\req_set.py", line 620, in _prepare_file session=self.session, hashes=hashes) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\download.py", line 821, in unpack_url hashes=hashes File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\download.py", line 659, in unpack_http_url hashes) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\download.py", line 882, in _download_http_url _download_url(resp, link, content_file, hashes) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\download.py", line 603, in _download_url hashes.check_against_chunks(downloaded_chunks) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\utils\hashes.py", line 46, in check_against_chunks for chunk in chunks: File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\download.py", line 571, in written_chunks for chunk in chunks: File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\utils\ui.py", line 141, in iter self.next(n) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\_vendor\progress\__init__.py", line 73, in next self.update() File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\_vendor\progress\bar.py", line 79, in update self.writeln(line) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\_vendor\progress\helpers.py", line 68, in writeln print(line, end='', file=self.file) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\_vendor\colorama\ansitowin32.py", line 141, in write self.write_and_convert(text) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\_vendor\colorama\ansitowin32.py", line 169, in write_and_convert self.write_plain_text(text, cursor, len(text)) File "C:\Users\Smith\Anaconda2\envs\server_monitor\lib\site-packages\pip\_vendor\colorama\ansitowin32.py", line 175, in write_plain_text self.wrapped.flush() OSError: raw write() returned invalid length 180 (should have been between 0 and 90) 33% |ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûè | 1.8MB 2.3MB/s eta 0:00:02
I assume it has something to do with the reference to cursor, but I'm not actually going to bother trying to figure out for sure because the install works just fine in my normal command prompt:
cd project_dir pip install -e .
Now we install the testing dependencies because we're good developers who test things:
pip install -e ".[testing]"
And now we can run our tests using:
Which runs the two sample tests that were created by our cookiecutter template.
We can also start our project using pserve and our development configuration:
This launches the development web server. But when I navigate over to it in my browser, I'm told I need to do more stuff first:
Pyramid is having a problem using your SQL database. The problem might be caused by one of the following things: 1. You may need to run the "initialize_server_monitor_db" script to initialize your database tables. Check your virtual environment's "bin" directory for this script and try to run it. 2. Your database server may not be running. Check that the database server referred to by the "sqlalchemy.url" setting in your "development.ini" file is running. After you fix the problem, please restart the Pyramid application to try it again.
On Windows, this is a little annoying because the database initialization executable is in the directory of my virtual environment but I need to pass it the development.ini file as an argument:
Cool, now we can actually start our server.
Next, I'll actually write some code myself.