Commit a4532560 authored by Alex's avatar Alex Committed by Madi Baig
Browse files

Version 1.0.1

parents
Pipeline #218207 failed with stages
in 1 minute and 1 second
**/__pycache__/**
stages:
- tests
- deploy
testing:
image: "python:3.7"
stage: tests
before_script:
- apt-get update -y
- apt-get install -y libsasl2-dev libldap2-dev libssl-dev
- python -m pip install -r requirements.txt
script:
- python -m pytest tests
deploy:
stage: deploy
only:
- master
script:
- chmod +x ./setup_push.sh
- ./setup_push.sh ssh://dokku@$VM_1:22/$APP_NAME master
- if [ -z ${VM_2+x} ]; then echo "no VM_2 in this group"; else ./setup_push.sh ssh://dokku@$VM_2:22/$APP_NAME master; fi
web: gunicorn app:app --log-file=-
\ No newline at end of file
# Python3 Flask Template
A started template for Flask web apps using **Python 3.8**. This started template includes:
- Dynamic frontend templates (not React)
- Database support
- Imperial LDAP user authentication
- Multiple environments (development and production)
- DoC Materials and Imperial Panopto API functionality
This template is useful if you would like to create a backend REST API, optionally coupled with a simple dynamic frontend.
## Getting started
Once you have created your new app, take a few minutes to look through the files to familiarise yourself with the project structure.
- `app.py` : entry point to the Flask app
- `templates/` : contains the frontend dynamic HTML files
- `static/` : contains the static frontend assets (images, stylesheets and scripts)
- `blueprints/` : contains all the application routes
- `models/` : contains all the database models/schemas
- `database/` : contains the database creation
- `config/` : contains the app settings for the different environments
**The first change you should make** is to set the `APP_NAME` variable in `config/config.py` to whatever you app's name is.
To start the application locally, you can just run `python3 app.py` and this will launch the app on port 5000 (by default).
You will notice a message in the console saying:
`WARNING: Could not connect to the given database URL!`
To fix this, you should set the environment variable DATABASE_URL accordingly. If you have PostgreSQL running locally, you can use that. Alternatively, you could use SQLite which is much simpler and does not require installation, for example, by running ```export DATABASE_URL="sqlite:///dev.db"```
If you do not want to use a database yet, you can ignore this warning and delete any routes that interact with the database.
If you navigate to `http://localhost:5000`, you will see the response created by the route defined in `blueprints/home.py`.
You will also notice the lines `Environment: production` and `Debug mode: off` when the Flask application starts in the console. To enable debug mode, you must set the environment variable `ENV` to `dev`, ie: `export ENV=dev` (see `config/config.py` for more details on different environments).
## Tutorial 1: Adding a new route
In this template, we have provided several routes in the ```blueprints/``` directory. For example, we have provided all authentication-related routes (such as login and logout) in the ```auth.py```. It is good practice to group your routes logically and create new blueprints if necessary.
To create a new blueprint, create a new file with the name of your blueprint and add:
```python
from flask import Blueprint
my_new_blueprint = Blueprint("new_bp", __name__, url_prefix="/new_bp")
```
You can then add a new route to that blueprint as follows:
```python
@my_new_blueprint.route("/hello")
def hello():
return "Hello!"
```
Finally and importantly, you must **register the blueprint** with the Flask application. To do this, go to ```app.py``` and register the blueprint in the same style the existing code does so.
Your new route will now be accessible at ```/new_bp/hello```.
## Tutorial 2: Adding database interaction
An important part of web app development is adding database support. If you ticked the "Include DB" option when creating the app, you will have automatically received a PostgreSQL database. You can get the connection string on the dashboard and connecting to the database using it to verify that you do indeed have a database.
For local development, you should either use an SQLite db or install PostgreSQL, and set the ```DATABASE_URL``` environment variable as explained in the Getting Started section.
### Defining database schemas
When working with relational databases, you must define **schemas** (models) for your tables. For example, if you are developing a web app where users can write blog posts, you may want to define a schema for a blog post and a user.
To do this, you can create new schemas/models in the ```models/``` directory and create a new file. In this template, we have an ```Entity``` model which has a primary key ID, a name which is a string and an age which is an int.
For the blog example, let's create a schema for a blog post. Create a file called ```post.py``` and add:
```python
from database.db import db
class Post(db.Model):
id = db.Column(db.Integer, primary_key=True)
title = db.Column(db.String(120), unique=False, nullable=False)
text = db.Column(db.Test, unique=False, nullable=False)
```
You then need to make sure the database will create the relevant tables in the database. Go to ```app.py``` and import your model here:
```python
# Import all your database models just like below:
with app.app_context():
from models.entity import Entity
if app.config["SQLALCHEMY_DATABASE_URI"]:
db.create_all()
```
For more on schemas using SQL-Alchemy, [read here](https://flask-sqlalchemy.palletsprojects.com/en/2.x/models/).
### Querying the database
You can now use your database to create, read, update or delete records using the SQL-Alchemy package. For example, if we want to create a new Post and add it to the database, we can do something like:
```python
from database.db import db
post = db.session.query(Post).filter_by(title="My article").first()
new_post = Post(title="A new article!", text="This is my amazing article, I hope you enjoyed it.")
db.session.add(new_post)
db.session.commit()
```
For more examples, please read around the [SQL-Alchemy package](https://flask-sqlalchemy.palletsprojects.com/en/2.x/).
### Running migrations and updating schemas
To migrate your production database, you should connect to it directly using the connection string available in the PaaS dashboard with a tool like [PgAdmin](https://www.pgadmin.org/) (or connecting to it using your terminal), and run the relevant queries there.
## Tutorial 3: Configuring a test environment
Often, we want to test that our web application is working as expected by writing a series of tests. For a simple example, you can view the code in the ```tests/``` directory.
Unit testing a web app can be difficult, so we can test simple behaviours such as checking that a particular route is working as expected. We can setup a dummy app using a test configuration, as demonstrated in the ```test_home.py``` file. You can add more tests in this directory by making new functions with the ```test_``` prefix.
To run these tests, you can run ```python -m pytest tests``` from the root directory. You should always run these tests before pushing/deploying new code, as they can catch out bugs. In fact, the CI/CD pipeline will run the tests for you, meaning that if a test fails, the pipeline will stop and your code won't be deployed until you fix the code causing the tests to fail.
from .app import app
\ No newline at end of file
from flask import Flask, send_from_directory
from database.db import db
from config.config import ENV, get_app_config, get_static_url
# Create and configure our Flask app
app = Flask(__name__, static_url_path=get_static_url())
app.url_map.strict_slashes = False
app.config.from_object(get_app_config())
# Setup authentication
import imperial_ldap
imperial_ldap.init_app(app)
imperial_ldap.set_auth_type(imperial_ldap.config.AuthType.SESSION, redirect_url='auth.login')
# Add integration to the Materials API
import imperial_doc_materials
imperial_doc_materials.init_app(app)
imperial_doc_materials.set_redirect_url('auth.login')
# Setup panopto api
import imperial_panopto
imperial_panopto.init_app(app)
imperial_panopto.set_client_id("<SET YOUR CLIENT ID HERE>")
imperial_panopto.set_client_secret("<SET YOUR CLIENT SECRET HERE>")
app.register_blueprint(imperial_panopto.panopto_api_blueprint)
imperial_panopto.set_panopto_is_dev(ENV != 'prod')
db.init_app(app)
# Import all your database models just like below:
with app.app_context():
from models.entity import Entity
if app.config["SQLALCHEMY_DATABASE_URI"]:
db.create_all()
# Serve all static assets for the frontend
@app.route('/static/<path:path>')
def serve_static_files(path):
return send_from_directory('static', path)
# Register all routes from the blueprints module
from blueprints.home import home_blueprint
from blueprints.auth import auth_blueprint
app.register_blueprint(home_blueprint)
app.register_blueprint(auth_blueprint)
# Hook any custom Jinja templating functions
from config import CUSTOM_TEMPLATE_FUNCTIONS
app.jinja_env.globals.update(CUSTOM_TEMPLATE_FUNCTIONS)
if __name__ == '__main__':
app.run()
\ No newline at end of file
from flask import Blueprint, render_template, request, redirect, flash
from imperial_ldap.auth import ldap_login, ldap_logout
from config import url_for2
from imperial_doc_materials import materials_login
auth_blueprint = Blueprint("auth", __name__, url_prefix="/auth")
@auth_blueprint.route("/login", methods=["GET", "POST"])
def login():
if request.method == "GET":
return render_template("login.html")
# Handle post request
username = request.form.get("username")
password = request.form.get("password")
if username and password:
user = ldap_login(username, password)
if user:
# We're logged in here, so now we also login into the Material API
materials_login(username, password)
return redirect(url_for2("home.dashboard"))
else:
# bad credentials
flash("Invalid login credentials.")
return redirect(url_for2("auth.login"))
return "Please provide a username and password"
@auth_blueprint.route("/logout", methods=["GET"])
def logout():
ldap_logout()
return redirect(url_for2("home.index"))
from flask import Blueprint, render_template, request, redirect
from imperial_panopto import panopto_login_required
from config import url_for2
from config.config import ENV, APP_NAME
from database.db import db
from models.entity import Entity
from imperial_ldap.auth import login_required, get_user_from_session
from imperial_doc_materials import using_materials
home_blueprint = Blueprint('home', __name__, url_prefix='/')
@home_blueprint.route('')
def index():
user = get_user_from_session() # If there is no user logged, this returns None
return render_template('index.html', user=user, dev=ENV, appname=APP_NAME)
@home_blueprint.route('/dashboard')
@login_required # Indicate that the user must be logged in (via LDAP)
@using_materials # Indicate that we want to access the Materials API
def dashboard(materials_client, user):
my_courses = materials_client.get_courses_for_year("2122")
return render_template('dashboard.html', user=user, my_courses=my_courses, dev=ENV, appname=APP_NAME)
@home_blueprint.route('/panopto_example')
@panopto_login_required
def panopto_example(panopto_client):
video = panopto_client.get_partially_viewed_videos()[0]
return render_template('panopto_example.html', video=video, iframe=panopto_client.get_iframe_of_video(video.id), dev=ENV, appname=APP_NAME)
@home_blueprint.route('/entities', methods=['GET', 'POST'])
@login_required
def entities(user):
if request.method == "GET":
try:
rows = ""
for e in db.session.query(Entity).all():
rows += str(e) + "<br>"
return rows
except:
return "App database error"
else:
name = request.form.get('name')
age = request.form.get('age')
try:
db.session.add(Entity(name=name, age=int(age)))
db.session.commit()
return redirect(url_for2('.entities'))
except:
return "Could not add entity."
# Define any custom Jinja2 functions here
from flask import url_for
from .config import URL_PREFIX
# Get around the routing prefix issue inside the templates
def url_for2(endpoint: str, **kwargs):
return URL_PREFIX + str(url_for(endpoint, **kwargs))
CUSTOM_TEMPLATE_FUNCTIONS = {
"url": url_for2,
}
import os
# This should match exactly the name of the app you specified
APP_NAME = "MY_APP_NAME_HERE"
ENV = os.environ.get('ENV', 'prod').lower()
URL_PREFIX = f"/{APP_NAME}" if ENV == 'prod' else ""
# Get the static URL of the app (to get around the production path issue)
def get_static_url():
if ENV == 'prod':
return f'/{APP_NAME}/static'
else:
return '/static'
# Get the app configuration based on the ENV environment variable (default is prod)
def get_app_config():
if ENV == 'prod':
return ProductionConfig()
else:
return DevelopmentConfig()
# If you have created a database for this app, the connection string will be automatically
# accessible through the DATABASE_URL environment variable.
def get_db_url(testing=False):
if testing:
return 'sqlite://{0}/{1}.db'.format("tmp", "testing")
url = os.environ.get('DATABASE_URL')
if url is None:
print("WARNING: Could not connect to the given database URL!")
# For PostgreSQL databases, the conn string needs to start with "postgresql"
if url and url.startswith("postgres://"):
url = url.replace("postgres://", "postgresql://", 1)
return url
class CommonConfig:
APP_NAME = APP_NAME
URL_PREFIX = URL_PREFIX
SECRET_KEY = "my-secrefasdfasdfasdfasdt-key"
SESSION_COOKIE_SECURE = True
SQLALCHEMY_DATABASE_URI = get_db_url()
SQLALCHEMY_TRACK_MODIFICATIONS = False
# Flask App settings for production environment
class ProductionConfig(CommonConfig):
DEBUG = False
APPLICATION_ROOT = f"/{APP_NAME}"
# Flask App settings for local development enviroment
class DevelopmentConfig(CommonConfig):
DEBUG = True
# Settings for running tests
class TestConfig(CommonConfig):
TESTING = True
SQLALCHEMY_DATABASE_URI = get_db_url(testing=True)
\ No newline at end of file
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
\ No newline at end of file
from database.db import db
class Entity(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(80), unique=False, nullable=False)
age = db.Column(db.Integer, unique=False, nullable=False)
def __repr__(self):
return f'Entity({self.id}, {self.name}, {self.age})'
python-3.8.12
#!/bin/bash -e
url=$1
branch=$2
if [ -z "$SSH_PRIVATE_KEY" ]; then
>&2 echo "Set SSH_PRIVATE_KEY environment variable"
exit 1
fi
ssh_host=$(echo $url | sed 's/.*@//' | sed 's/[:/].*//')
if [ -z "$ssh_host" ]; then
>&2 echo "Usage: $0 <user@git.host:project | ssh://user@git.host:port/project> [<branch>]"
exit 1
fi
ssh_port=
if [[ $url =~ ^ssh://[^/]+:([0-9]+) ]]; then
ssh_port="-p ${BASH_REMATCH[1]}"
fi
# TODO: skip on multiple runs
mkdir -p ~/.ssh
echo "$SSH_PRIVATE_KEY" | tr -d '\r' > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
ssh-keyscan -H $ssh_port "$ssh_host" >> ~/.ssh/known_hosts
git push $url ${CI_COMMIT_SHA:-HEAD}:refs/heads/${branch:-master} $([ -z "$DISABLE_FORCE_PUSH" ] && echo --force)
#content {
margin-left: 1%;
margin-right: 1%;
}
\ No newline at end of file
html,
body {
height: 100%;
}
body {
display: -ms-flexbox;
display: -webkit-box;
display: flex;
-ms-flex-align: center;
-ms-flex-pack: center;
-webkit-box-align: center;
align-items: center;
-webkit-box-pack: center;
justify-content: center;
padding-top: 40px;
padding-bottom: 40px;
background-color: #f5f5f5;
}
.form-signin {
width: 100%;
max-width: 330px;
padding: 15px;
margin: 0 auto;
}
.form-signin .checkbox {
font-weight: 400;
}
.form-signin .form-control {
position: relative;
box-sizing: border-box;
height: auto;
padding: 10px;
font-size: 16px;
}
.form-signin .form-control:focus {
z-index: 2;
}
.form-signin input[type="email"] {
margin-bottom: -1px;
border-bottom-right-radius: 0;
border-bottom-left-radius: 0;
}
.form-signin input[type="password"] {
margin-bottom: 10px;
border-top-left-radius: 0;
border-top-right-radius: 0;
}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment