Merge pull request #184 from dsgnr/new_api

Refactor both front-end and backend to make app more simple and portable

.gitignore

@@ -139,7 +139,7 @@ cython_debug/
 
 # Project
 .DS_Store
-webui/node_modules
+node_modules
 k8s.yml
 
 # Elastic Beanstalk Files

CHANGELOG.md

@@ -2,6 +2,25 @@
 
 All notable changes to this project will be documented in this file.
 
+## [2.0.0b] - 2023-07-23
+
+Rewrites the front-end to a single page HTML instead of React.
+This is an incredibly small and simple application, so maintaining
+the various dependencies that come with a React site is a bit overkill.
+
+The backend API is also refactored to move away from using AWS Lambda functions.
+The usage and popularity of this site is absolutely fantastic, but also quite unexpected.
+There have been numerous overage fees within the AWS free-tier limits,
+which is not sustainable without donations from users.
+
+I see discussions related to portchecker.io, and some users have asked about self hosting.
+The changes made here also means that this project is more portable/not vendor locked
+in and only requires Docker to run.
+
+As always, feedback and recommendations are more than welcomed!
+
+Please consider the rewritten front-end and API as a beta release.
+
 ## [1.0.11] - 2021-12-27
 
 - Improve various UI elements for better experience

README.md

@@ -4,28 +4,55 @@
 
 This project aims to be a simple, go-to place for querying the port status for a provided hostname or IP address.
 
-The front-end is built using React, and is forked from [ofnullable/react-spa-template](https://github.com/ofnullable/react-spa-template).
-
-## 📝 To Do
-
-- [x] Improve mobile browsing style
-- [ ] Document API route
-- [ ] More tests
-- [x] Implement front-end
-- [ ] SEO/search engine submissions
-- [ ] Bugfix/security
 
 ### 🏠 [Homepage](https://portchecker.io)
 
 ### ✨ [Demo](https://portchecker.io)
 
+### API Specs
+The API documentation is automatically generated by FastAPI.
+Routes and specification can be found at [https://portchecker.io/api/v1/docs](https://portchecker.io/api/v1/docs)
+
 ## Author
 
 👤 **Dan Hand**
 
 * Website: https://danielhand.io
 * Github: [@dsgnr](https://github.com/dsgnr)
 
+### Getting Started
+The project consists of two containers. The front-end is a static HTML file sat behind Nginx. The back-end is a simple API built using FastAPI.
+
+The project aims to be super simple, with low overhead and also the least amount of dependencies as possible.
+
+The project contains both production and development stacks. The production stack utilises Gunicorn as the API's gateway interface, whereas development utilises `uvicorn`.
+
+Bringing up the development stack:
+~~~
+$ docker-compose -f docker-compose-dev.yml up  --build
+~~~
+
+Bringing up the API outside of Docker;
+~~~
+$ cd backend/api; uvicorn main:app --reload
+~~~
+
+Bringing up the UI outside of Docker;
+~~~
+$ cd frontend/web; yarn install; yarn dev
+~~~
+
+The port checker API will be running at [http://0.0.0.0:8000](http://0.0.0.0:8000) and the front-end will be running at [http://0.0.0.0:8080](http://0.0.0.0:8080).
+
+## Configuration
+The following configuration options are available. These would be set within the Docker compose files.
+
+### API:
+- ALLOW_PRIVATE (boolean, default false)
+
+### UI:
+- DEFAULT_PORT (integer, default 443)
+
 ## 🤝 Contributing
 
 Contributions, issues and feature requests are welcome.<br />
@@ -52,3 +79,4 @@ This project is [MIT](https://github.com/kefranabg/readme-md-generator/blob/mast
 ---
 ***
 _This README was generated with ❤️ by [readme-md-generator](https://github.com/kefranabg/readme-md-generator)_
+

backend/Dockerfile

@@ -0,0 +1,10 @@
+FROM python:3.11-slim
+
+COPY requirements.txt entrypoint.sh /
+COPY api /src
+RUN chmod +x /entrypoint.sh
+RUN pip3 install --no-cache-dir --upgrade -r /requirements.txt
+WORKDIR /src
+EXPOSE 8000
+CMD ["/entrypoint.sh"]
+

backend/Dockerfile.dev

@@ -0,0 +1,10 @@
+FROM python:3.11-slim
+
+COPY requirements.txt entrypoint_dev.sh /
+COPY api /src
+RUN chmod +x /entrypoint_dev.sh
+RUN pip3 install --no-cache-dir --upgrade -r /requirements.txt
+WORKDIR /src
+EXPOSE 8000
+CMD ["/entrypoint_dev.sh"]
+

backend/api/app/helpers/query.py

@@ -0,0 +1,54 @@
+# Standard Library
+import os
+import socket
+from ipaddress import ip_address
+from urllib.parse import urlparse
+
+
+def validate_port(port: int) -> bool:
+    return port in range(1, 65535 + 1)
+
+
+def is_ip_address(address: str) -> bool:
+    try:
+        return bool(ip_address(address))
+    except ValueError:
+        return False
+
+
+def is_address_valid(address: str) -> bool:
+    address_obj = ip_address(address)
+    if address_obj.is_private and not os.environ.get("ALLOW_PRIVATE"):
+        raise ValueError(
+            f"IPv{address_obj.version} address '{address}' does not appear to be public"
+        )
+    return address_obj.version
+
+
+def is_valid_hostname(hostname):
+    try:
+        parsed = urlparse(hostname)
+        if parsed.scheme:
+            raise ValueError("The hostname must not have a scheme")
+    except Exception as ex:
+        raise Exception(str(ex))
+
+    try:
+        socket.gethostbyname(hostname)
+        return True
+    except socket.gaierror:
+        raise Exception("Hostname does not appear to resolve")
+
+
+def query_ipv4(address, ports):
+    results = []
+    for port in ports:
+        result = {"port": port, "status": False}
+        sock = socket.socket()
+        sock.settimeout(1)
+        port_check = sock.connect_ex((address, int(port)))
+        if port_check == 0:
+            result["status"] = True
+        sock.close()
+        results.append(result)
+    return results

backend/api/app/routes/admin.py

@@ -0,0 +1,15 @@
+"""
+The API routes for admin
+"""
+# Third Party
+from fastapi.routing import APIRouter
+
+router = APIRouter()
+
+
+@router.get("/healthz")
+def health() -> bool:
+    """
+    Basic health check to ensure API is responding. Returns `True`.
+    """
+    return True

backend/api/app/routes/v1.py

@@ -0,0 +1,69 @@
+"""
+The API routes for V1
+"""
+# Third Party
+from app.helpers.query import (
+    is_address_valid,
+    is_ip_address,
+    is_valid_hostname,
+    query_ipv4,
+    validate_port,
+)
+from app.schemas.api import APIResponseSchema, APISchema
+from fastapi.responses import JSONResponse
+from fastapi.routing import APIRouter
+from fastapi_versioning import version
+
+router = APIRouter()
+
+
+@router.post("/query", response_model=APIResponseSchema)
+@version(1)
+def query_host(body: APISchema) -> APIResponseSchema:
+    ret = {"error": False, "host": None, "check": [], "msg": None}
+
+    try:
+        ret["host"] = body.host
+    except Exception:
+        ret["error"] = True
+        ret["msg"] = "A host must be defined"
+        return JSONResponse(status_code=400, content=ret)
+
+    try:
+        ports = body.ports
+    except Exception:
+        ret["error"] = True
+        ret["msg"] = "A list of ports must be defined"
+        return JSONResponse(status_code=400, content=ret)
+
+    try:
+        for port in ports:
+            if not validate_port(port):
+                raise ValueError(
+                    "Only a valid port number between 1 and 65535 can be queried. "
+                    f"Port {port} is not valid"
+                )
+    except Exception as ex:
+        ret["error"] = True
+        ret["msg"] = str(ex)
+        return JSONResponse(status_code=400, content=ret)
+
+    is_ip = is_ip_address(ret["host"])
+    ip_version = 4
+    try:
+        if is_ip:
+            ip_version = is_address_valid(ret["host"])
+        else:
+            is_valid_hostname(ret["host"])
+    except Exception as ex:
+        ret["error"] = True
+        ret["msg"] = str(ex)
+        return JSONResponse(status_code=400, content=ret)
+
+    if ip_version == 6:
+        ret["error"] = True
+        ret["msg"] = "IPv6 is not currently supported"
+        return JSONResponse(status_code=400, content=ret)
+
+    ret["check"] = query_ipv4(ret["host"], ports)
+    return JSONResponse(status_code=200, content=ret)

backend/api/app/schemas/api.py

@@ -0,0 +1,41 @@
+"""
+The API schema for V1
+"""
+# Standard Library
+from ipaddress import IPv4Address
+from typing import List, Union
+
+# Third Party
+from pydantic import BaseModel, Field
+
+
+class APISchema(BaseModel):
+    host: Union[IPv4Address, str] = Field(description="The IPv4 address of the host to query")
+    ports: List[int]
+
+    model_config = {"json_schema_extra": {"examples": [{"host": "1.1.1.1", "ports": [444]}]}}
+
+
+class APICheckSchema(BaseModel):
+    port: int
+    status: bool
+
+
+class APIResponseSchema(BaseModel):
+    error: bool
+    msg: Union[str, None]
+    check: List[APICheckSchema]
+    host: str
+
+    model_config = {
+        "json_schema_extra": {
+            "examples": [
+                {
+                    "error": False,
+                    "msg": None,
+                    "host": "1.1.1.1",
+                    "check": [{"status": True, "ports": 443}],
+                }
+            ]
+        }
+    }

backend/api/main.py

@@ -0,0 +1,29 @@
+# Standard Library
+import logging
+
+# Third Party
+from fastapi import FastAPI
+from fastapi.logger import logger as fastapi_logger
+from fastapi.middleware.cors import CORSMiddleware
+from fastapi_versioning import VersionedFastAPI
+
+# First Party
+from app.routes import admin, v1
+
+# Init the app
+app = FastAPI(title="portchecker.io", version="1.0.0")
+
+# Logging
+gunicorn_error_logger = logging.getLogger("gunicorn.error")
+gunicorn_logger = logging.getLogger("gunicorn")
+uvicorn_access_logger = logging.getLogger("uvicorn.access")
+uvicorn_access_logger.handlers = gunicorn_error_logger.handlers
+fastapi_logger.handlers = gunicorn_error_logger.handlers
+fastapi_logger.setLevel(gunicorn_logger.level)
+
+# Routes
+app.include_router(v1.router, tags=["Routes"])
+app = VersionedFastAPI(app, version_format="{major}", prefix_format="/api/v{major}")
+app.include_router(admin.router, tags=["Admin"])
+
+app.add_middleware(CORSMiddleware, allow_origins=["*"], allow_methods=["*"], allow_headers=["*"])

backend/entrypoint.sh

@@ -0,0 +1,6 @@
+#! /usr/bin/env sh
+set -e
+exec gunicorn -k uvicorn.workers.UvicornWorker \
+    -b 0.0.0.0:8000 \
+    --workers 4 \
+    main:app

backend/entrypoint_dev.sh

@@ -0,0 +1,4 @@
+#! /usr/bin/env sh
+set -e
+exec uvicorn main:app --host 0.0.0.0 --port 8000
+

backend/pyproject.toml

@@ -0,0 +1,19 @@
+[tool.black]
+line-length = 100
+target-version = ["py39"]
+skip-string-normalization = false
+skip-numeric-underscore-normalization = false
+
+[tool.isort]
+combine_as_imports = true
+force_grid_wrap = 0
+include_trailing_comma = true
+multi_line_output = 3
+use_parentheses = true
+line_length = 100
+wrap_length = 100
+ensure_newline_before_comments = true
+
+import_heading_firstparty = "First Party"
+import_heading_stdlib = "Standard Library"
+import_heading_thirdparty = "Third Party"