Practical application — Build an FTP server from scratch

Note

This page uses two different API variants:

• Synchronous API with classic def functions, usable in any context.

• Asynchronous API with async def functions, using an asynchronous framework to perform I/O operations.

All asynchronous API examples assume that you are using asyncio, but you can use a different library thanks to the asynchronous backend engine API.

---

TL;DR

Yes, I know, you will never need to create your own FTP server (unless you want your own service). However, it is still interesting to see the structure of such a model, based on a standardized communication protocol.

The File Transfer Protocol (as defined in RFC 959) is a good example of how to set up a server with precise rules.

We are not going to implement all the requests (that is not the point). The tutorial will show you how to set up the infrastructure and exploit all (or most) of the EasyNetwork library’s features.

The Communication Protocol

FTP requests and responses are transmitted as ASCII strings separated by a carriage return (\r\n).

Let’s say we want to have two classes FTPRequest and FTPReply to manage them in our request handler.

FTPRequest Object

An FTP client request consists of a command and, optionally, arguments separated by a space character.

First, we define the exhaustive list of available commands (c.f. RFC 959 (Section 4.1)):

ftp_command.py

from __future__ import annotations

from enum import StrEnum, auto


class FTPCommand(StrEnum):
    """This is an enumeration of all the commands defined in RFC 959.

    If the client sends a command that is not one of these,
    the server will reply a 500 error code.

    If the server has not implemented one of these commands,
    it will reply a 502 error code.
    """

    @staticmethod
    def _generate_next_value_(
        name: str,
        start: int,
        count: int,
        last_values: list[str],
    ) -> str:
        assert name.isupper()
        return name

    ABOR = auto()
    ACCT = auto()
    ALLO = auto()
    APPE = auto()
    CDUP = auto()
    CWD = auto()
    DELE = auto()
    HELP = auto()
    LIST = auto()
    MKD = auto()
    MODE = auto()
    NOOP = auto()
    PASS = auto()
    PASV = auto()
    PORT = auto()
    PWD = auto()
    QUIT = auto()
    REIN = auto()
    REST = auto()
    RETR = auto()
    RMD = auto()
    RNFR = auto()
    RNTO = auto()
    SITE = auto()
    SMNT = auto()
    STAT = auto()
    STOR = auto()
    STOU = auto()
    STRU = auto()
    SYST = auto()
    TYPE = auto()
    USER = auto()

Note

See enum module documentation to understand the usage of auto and _generate_next_value_().

Second, we define the FTPRequest class that will be used:

ftp_request.py

from __future__ import annotations

from dataclasses import dataclass
from typing import Any

from ftp_command import FTPCommand


@dataclass(frozen=True, match_args=True)
class FTPRequest:
    """Dataclass defining an FTP request"""

    command: FTPCommand
    """Command name."""

    args: tuple[Any, ...]
    """Command arguments sequence. May be empty."""

FTPReply Object

An FTP reply consists of a three-digit number (transmitted as three alphanumeric characters) followed by some text.

ftp_reply.py

from __future__ import annotations

from dataclasses import dataclass


@dataclass(frozen=True)
class FTPReply:
    """Dataclass defining an FTP reply."""

    code: int
    """The reply code."""

    message: str
    """A line of text following the reply code."""

Use Converters To Handle Character Strings

The client will send a character string and expect a character string in return. StringLineSerializer will handle this part, but we have created our objects in order not to manipulate strings.

To remedy this, we will use converters to switch between our FTPRequest / FTPReply objects and strings.

ftp_converters.py

from __future__ import annotations

from easynetwork.converter import AbstractPacketConverter
from easynetwork.exceptions import PacketConversionError

from ftp_command import FTPCommand
from ftp_reply import FTPReply
from ftp_request import FTPRequest


class FTPRequestConverter(AbstractPacketConverter[FTPRequest, str]):
    """Converter to switch between FTPRequest objects and strings."""

    def convert_to_dto_packet(self, obj: FTPRequest) -> str:
        """Creates the string representation of the FTPRequest object

        Not implemented.
        """

        raise NotImplementedError("Not needed in server side")

    def create_from_dto_packet(self, packet: str) -> FTPRequest:
        """Builds an FTPRequest object from a raw string

        >>> c = FTPRequestConverter()
        >>> c.create_from_dto_packet("NOOP")
        FTPRequest(command=<FTPCommand.NOOP: 'NOOP'>, args=())
        >>> c.create_from_dto_packet("qUiT")
        FTPRequest(command=<FTPCommand.QUIT: 'QUIT'>, args=())
        >>> c.create_from_dto_packet("STOR /path/file.txt")
        FTPRequest(command=<FTPCommand.STOR: 'STOR'>, args=('/path/file.txt',))
        >>> c.create_from_dto_packet("invalid command")
        Traceback (most recent call last):
        ...
        easynetwork.exceptions.PacketConversionError: Command unrecognized: 'INVALID'

        Parameters:
            packet: The string representation of the request

        Returns:
            the FTP request
        """
        command, *args = packet.split(" ")
        command = command.upper()
        try:
            command = FTPCommand(command)
        except ValueError as exc:
            raise PacketConversionError(f"Command unrecognized: {command!r}") from exc
        return FTPRequest(command, tuple(args))


class FTPReplyConverter(AbstractPacketConverter[FTPReply, str]):
    """Converter to switch between FTPReply objects and strings."""

    def convert_to_dto_packet(self, obj: FTPReply) -> str:
        """Creates the string representation of the FTPReply object

        >>> c = FTPReplyConverter()
        >>> c.convert_to_dto_packet(FTPReply(200, "Command okay."))
        '200 Command okay.'
        >>> c.convert_to_dto_packet(FTPReply(10, "Does not exist but why not."))
        '010 Does not exist but why not.'

        Parameters:
            obj: The FTPReply object

        Returns:
            the string representation of the reply
        """

        code: int = obj.code
        message: str = obj.message

        assert 0 <= code < 1000, f"Invalid reply code {code:d}"

        # Multi-line replies exist, but we won't deal with them in this tutorial.
        assert "\n" not in message, "message contains newline character"
        separator = " "

        return f"{code:03d}{separator}{message}"

    def create_from_dto_packet(self, packet: str) -> FTPReply:
        """Builds an FTPReply object from a raw string

        Not implemented.
        """
        raise NotImplementedError("Not needed in server side")

Note

In FTPRequestConverter.create_from_dto_packet(), the arguments are left as sent and returned.

try:
    command = FTPCommand(command)
except ValueError as exc:
    raise PacketConversionError(f"Command unrecognized: {command!r}") from exc
return FTPRequest(command, tuple(args))

An improvement would be to process them here and not leave the job to the request handler. But since we are not building a real (complete and fully featured) FTP server, we will leave the code as is.

The Protocol Object

Now that we have our business objects, we can create our protocol object.

ftp_server_protocol.py

from __future__ import annotations

from easynetwork.converter import StapledPacketConverter
from easynetwork.protocol import StreamProtocol
from easynetwork.serializers import StringLineSerializer

from ftp_converters import FTPReplyConverter, FTPRequestConverter
from ftp_reply import FTPReply
from ftp_request import FTPRequest


class FTPServerProtocol(StreamProtocol[FTPReply, FTPRequest]):
    def __init__(self) -> None:
        request_converter = FTPRequestConverter()
        response_converter = FTPReplyConverter()

        super().__init__(
            serializer=StringLineSerializer(newline="CRLF", encoding="ascii"),
            converter=StapledPacketConverter(
                sent_packet_converter=response_converter,
                received_packet_converter=request_converter,
            ),
        )

Note

Note the use of StapledPacketConverter:

super().__init__(
    serializer=StringLineSerializer(newline="CRLF", encoding="ascii"),
    converter=StapledPacketConverter(
        sent_packet_converter=response_converter,
        received_packet_converter=request_converter,
    ),
)

It will create a composite converter with our two converters.

The Server

FTPReply: Define Default Replies

A good way to reply to the client with default replies is to define them in methods.

Here are just a few that will be used in this tutorial.

ftp_reply.py

@dataclass(frozen=True)
class FTPReply:
    """Dataclass defining an FTP reply."""

    code: int
    """The reply code."""

    message: str
    """A line of text following the reply code."""

    @staticmethod
    def ok() -> FTPReply:
        return FTPReply(200, "Command okay.")

    @staticmethod
    def service_ready_for_new_user() -> FTPReply:
        return FTPReply(220, "Service ready for new user.")

    @staticmethod
    def connection_close(*, unexpected: bool = False) -> FTPReply:
        if unexpected:
            return FTPReply(421, "Service not available, closing control connection.")
        return FTPReply(221, "Service closing control connection.")

    @staticmethod
    def syntax_error() -> FTPReply:
        return FTPReply(500, "Syntax error, command unrecognized.")

    @staticmethod
    def not_implemented_error() -> FTPReply:
        return FTPReply(502, "Command not implemented.")

The Request Handler

Let’s create this request handler.

Service Initialization

A feature we could have used for the echo client/server over TCP tutorial is to define actions to perform at start/end of the server.

Here, we’ll only initialize the logger, but we could also use it to prepare the folders and files that the server should handle (location, permissions, file existence, etc.).

class FTPRequestHandler(AsyncStreamRequestHandler[FTPRequest, FTPReply]):
    async def service_init(
        self,
        exit_stack: contextlib.AsyncExitStack,
        server: Any,
    ) -> None:
        self.logger = logging.getLogger(self.__class__.__name__)

Control Connection Hooks

Here are the features brought by AsyncStreamRequestHandler: It is possible to perform actions when connecting/disconnecting the client.

async def on_connection(self, client: AsyncStreamClient[FTPReply]) -> None:
    await client.send_packet(FTPReply.service_ready_for_new_user())

async def on_disconnection(self, client: AsyncStreamClient[FTPReply]) -> None:
    with contextlib.suppress(ConnectionError):
        if not client.is_closing():
            await client.send_packet(FTPReply.connection_close(unexpected=True))

The handle() Method

Only NOOP and QUIT commands will be implemented for this tutorial. All parse errors are considered syntax errors.

async def handle(
    self,
    client: AsyncStreamClient[FTPReply],
) -> AsyncGenerator[None, FTPRequest]:
    client_address = client.extra(INETClientAttribute.remote_address)
    try:
        request: FTPRequest = yield
    except StreamProtocolParseError as exc:
        self.logger.warning(
            "%s: %s: %s",
            client_address,
            type(exc.error).__name__,
            exc.error,
        )
        await client.send_packet(FTPReply.syntax_error())
        return

    self.logger.info("Sent by client %s: %s", client_address, request)
    match request:
        case FTPRequest(FTPCommand.NOOP):
            await client.send_packet(FTPReply.ok())

        case FTPRequest(FTPCommand.QUIT):
            async with contextlib.aclosing(client):
                await client.send_packet(FTPReply.connection_close())

        case _:
            await client.send_packet(FTPReply.not_implemented_error())

Full Code

ftp_server_request_handler.py

from __future__ import annotations

import contextlib
import logging
from collections.abc import AsyncGenerator
from typing import Any

from easynetwork.api_async.server import AsyncStreamClient, AsyncStreamRequestHandler, INETClientAttribute
from easynetwork.exceptions import StreamProtocolParseError

from ftp_command import FTPCommand
from ftp_reply import FTPReply
from ftp_request import FTPRequest


class FTPRequestHandler(AsyncStreamRequestHandler[FTPRequest, FTPReply]):
    async def service_init(
        self,
        exit_stack: contextlib.AsyncExitStack,
        server: Any,
    ) -> None:
        self.logger = logging.getLogger(self.__class__.__name__)

    async def on_connection(self, client: AsyncStreamClient[FTPReply]) -> None:
        await client.send_packet(FTPReply.service_ready_for_new_user())

    async def on_disconnection(self, client: AsyncStreamClient[FTPReply]) -> None:
        with contextlib.suppress(ConnectionError):
            if not client.is_closing():
                await client.send_packet(FTPReply.connection_close(unexpected=True))

    async def handle(
        self,
        client: AsyncStreamClient[FTPReply],
    ) -> AsyncGenerator[None, FTPRequest]:
        client_address = client.extra(INETClientAttribute.remote_address)
        try:
            request: FTPRequest = yield
        except StreamProtocolParseError as exc:
            self.logger.warning(
                "%s: %s: %s",
                client_address,
                type(exc.error).__name__,
                exc.error,
            )
            await client.send_packet(FTPReply.syntax_error())
            return

        self.logger.info("Sent by client %s: %s", client_address, request)
        match request:
            case FTPRequest(FTPCommand.NOOP):
                await client.send_packet(FTPReply.ok())

            case FTPRequest(FTPCommand.QUIT):
                async with contextlib.aclosing(client):
                    await client.send_packet(FTPReply.connection_close())

            case _:
                await client.send_packet(FTPReply.not_implemented_error())

Start The Server

SynchronousAsynchronous

server.py

from __future__ import annotations

from collections.abc import Sequence

from easynetwork.api_sync.server import StandaloneTCPNetworkServer

from ftp_reply import FTPReply
from ftp_request import FTPRequest
from ftp_server_protocol import FTPServerProtocol
from ftp_server_request_handler import FTPRequestHandler


class FTPServer(StandaloneTCPNetworkServer[FTPRequest, FTPReply]):
    def __init__(
        self,
        host: str | Sequence[str] | None = None,
        port: int = 21000,
    ) -> None:
        super().__init__(host, port, FTPServerProtocol(), FTPRequestHandler())


if __name__ == "__main__":
    import logging

    def main() -> None:
        logging.basicConfig(
            level=logging.INFO,
            format="[ %(levelname)s ] [ %(name)s ] %(message)s",
        )
        with FTPServer() as server:
            try:
                server.serve_forever()
            except KeyboardInterrupt:
                pass

    main()

Outputs

The output of the example should look something like this:

Server:

IPv4 connectionIPv6 connection

(.venv) $ python server.py
[ INFO ] [ easynetwork.api_async.server.tcp ] Start serving at ('::', 21000), ('0.0.0.0', 21000)
[ INFO ] [ easynetwork.api_async.server.tcp ] Accepted new connection (address = ('127.0.0.1', 45994))
[ INFO ] [ FTPRequestHandler ] Sent by client ('127.0.0.1', 45994): FTPRequest(command=<FTPCommand.NOOP: 'NOOP'>, args=())
[ INFO ] [ FTPRequestHandler ] Sent by client ('127.0.0.1', 45994): FTPRequest(command=<FTPCommand.NOOP: 'NOOP'>, args=())
[ INFO ] [ FTPRequestHandler ] Sent by client ('127.0.0.1', 45994): FTPRequest(command=<FTPCommand.STOR: 'STOR'>, args=('/path/to/file.txt',))
[ WARNING ] [ FTPRequestHandler ] ('127.0.0.1', 45994): PacketConversionError: Command unrecognized: 'UNKNOWN'
[ INFO ] [ FTPRequestHandler ] Sent by client ('127.0.0.1', 45994): FTPRequest(command=<FTPCommand.QUIT: 'QUIT'>, args=())
[ INFO ] [ easynetwork.api_async.server.tcp ] ('127.0.0.1', 45994) disconnected

Client:

Note

The File Transfer Protocol is based on the Telnet protocol.

The telnet(1) command is used to communicate with another host using the Telnet protocol.

IPv4 connectionIPv6 connection

$ telnet -4 localhost 21000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 Service ready for new user.
NOOP
200 Command okay.
nOoP
200 Command okay.
STOR /path/to/file.txt
502 Command not implemented.
UNKNOWN command
500 Syntax error, command unrecognized.
QUIT
221 Service closing control connection.
Connection closed by foreign host.