notes/os/unix/admin/security/firewall/nft-nftables.txt

https://wiki.nftables.org/wiki-nftables/index.php/Main_Page
https://wiki.nftables.org/wiki-nftables/index.php/Sets
https://wiki.nftables.org/wiki-nftables/index.php/Quick_reference-nftables_in_10_minutes
https://wiki.nftables.org/wiki-nftables/index.php/Moving_from_iptables_to_nftables
https://wiki.gentoo.org/wiki/Nftables
https://wiki.archlinux.org/title/Nftables
    /etc/systemd/system/docker.service.d/netns.conf
        ... nsenter ...
https://wiki.archlinux.org/title/Nftables_(%D0%A0%D1%83%D1%81%D1%81%D0%BA%D0%B8%D0%B9)

https://wiki.nftables.org/wiki-nftables/index.php/Setting_packet_metainformation
    In this example, the conntrack mark is stored in the packet. 
        nft add rule filter forward meta mark set ct mark
https://wiki.nftables.org/wiki-nftables/index.php/Matching_packet_metainformation
    skuid <user id> 	UID associated with originating socket
    You can use your user name to match traffic, eg.
        nft add rule filter output meta skuid pablo[|1000] counter

    nft add rule filter output meta mark 123 counter
    secmark 	Y 	packet secmark 	integer (32 bit)

https://www.mankier.com/8/nft
https://www.netfilter.org/projects/nftables/manpage.html

tutorial
https://ral-arturo.org/2020/11/22/python-nftables-tutorial.html
https://github.com/aborrero/python-nftables-tutorial
https://people.netfilter.org/pablo/nft-tutorial.pdf
    ! p14
    nft add rule ip foo bar meta skuid 1000-1100
    nft add rule ip foo bar ct mark set 10
    nft add rule ip foo bar ct mark set meta mark

src
https://git.netfilter.org/nftables/
https://github.com/google/nftables

samples
https://kernelnewbies.org/nftables_examples

2023
LinuxCloudHacks - Nftables Firewall of p8
    https://www.youtube.com/playlist?list=PLUF494I4KUvqwDjhOoP3IFUpgEhE1OVDO
    Protecting Incoming Traffic with Nftables 3:00 of 24:34
        https://www.youtube.com/watch?v=K8JPwbcNy_0
        ! nft -i # interactive
2021
https://habr.com/ru/company/ruvds/blog/580648/
2020
https://www.redhat.com/en/blog/using-nftables-red-hat-enterprise-linux-8
    https://habr.com/ru/company/otus/blog/511122/
2016
iptables deprecation
    https://developers.redhat.com/blog/2016/10/28/what-comes-after-iptables-its-successor-of-course-nftables#known_limitations
    https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/9.0_release_notes/deprecated_functionality


/etc/nftables/
    main.nft
???
/etc/nftables.conf

nft 
    list ruleset

sudo systemctl status nftables.service
sudo systemctl is-enabled nftables.service

backend
    firewalld (https://wiki.archlinux.org/title/Firewalld)


sudo nft

table ip filter {
    chain output {
        type filter hook output priority 0; policy accept;
        ip daddr 0.0.0.0/0 meta skuid 1000($UID) mark set 0x1
    }
}

sudo nft list ruleset
sudo ip rule add fwmark 0x1 table 100
sudo ip route add default via 192.168.1.1 table 100

install packages
    nftables
    # cgroup-tools
    libcgroup-tools
        /etc/cgconfig.conf

sudo lsmod | grep net_cls
sudo mkdir /sys/fs/cgroup/net_cls/firefox
sudo sh -c "echo 0x100001 > /sys/fs/cgroup/net_cls/firefox/net_cls.classid"
sudo cgexec -g net_cls:firefox firefox

sudo nft add table ip filter
sudo nft add chain ip filter output { type filter hook output priority 0; }
sudo nft add rule ip filter output meta cgroup 0x100001 meta mark set 0x1

sudo nft list ruleset > /etc/nftables.conf
sudo systemctl enable nftables



*****************************************************************


# Создаем cgroup
mkdir -p /sys/fs/cgroup/my_cgroup
# Добавляем текущий процесс (или другой с PID x) в cgroup
echo $$ > /sys/fs/cgroup/my_cgroup/cgroup.procs

sudo nft add table inet filter
sudo nft add chain inet filter input { type filter hook input priority 0 \; }
???
sudo nft add rule  inet filter output cgroup v2path "/my_cgroup" meta mark set 0x1
sudo nft add rule  inet filter input cgroupv2 path /sys/fs/cgroup/my_cgroup mark set 0x1

sudo nft list ruleset
sudo tcpdump -nnvvv -i eth0 ip

https://git.netfilter.org/nftables/tree/py/src/nftables.py
https://blog.fraggod.net/2021/08/31/easy-control-over-applications-network-access-using-nftables-and-systemd-cgroup-v2-tree.html

!!!
https://unix.stackexchange.com/questions/752635/can-nftables-perform-postrouting-matching-on-crgroupv2
    !!! need to read
https://github.com/mk-fg/systemd-cgroup-nftables-policy-manager
    https://github.com/systemd/systemd/issues/7327

!!! man nft
https://man.archlinux.org/man/nft.8
https://netfilter.org/projects/nftables/manpage.html
META EXPRESSIONS
    meta {length | nfproto | l4proto | protocol | priority}
    [meta] {mark | ... | nftrace | rtclassid | ... | cgroup | ... }
    ...
    mark        Packet mark       mark
    rtclassid   Routing realm     realm
    cgroup      control group id  integer (32 bit)

SOCKET EXPRESSION
    socket {transparent | mark | wildcard}
    socket cgroupv2 level NUM   #(path from /sys/fs/cgroup)

https://bbs.archlinux.org/viewtopic.php?id=289682
    #
    nft add rule inet "${table_name}" output \
		    socket cgroupv2 level 5 \
		    \""${cgroup}"\" \
		    log
https://blog.fraggod.net/2021/08/31/easy-control-over-applications-network-access-using-nftables-and-systemd-cgroup-v2-tree.html
    https://github.com/mk-fg/systemd-cgroup-nftables-policy-manager
    https://github.com/mk-fg/systemd-cgroup-nftables-policy-manager/issues/2
    https://man.archlinux.org/man/systemd.resource-control.5
        NFTSet=[family]:[table]:[set]
        NFTSet=cgroup:inet:filter:my_service
        # This option is only available for system services and is not supported for services running in per-user instances of the service manager.
        # Added in version 255.
!!!
https://blogs.oracle.com/linux/post/cgroup-v2-meets-nftables
    https://github.com/libcgroup/libcgroup/issues/432
        ! workaround
    https://systemd.io/CGROUP_DELEGATION/

!!! cgroups v2
https://manpath.be/f35/7/cgroups
https://docs.kernel.org/admin-guide/cgroup-v2.html
https://www.kernel.org/doc/Documentation/cgroup-v2.txt
https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git/tree/Documentation/admin-guide/cgroup-v2.rst
https://zhiwayzhang.github.io/posts/cgroups/
https://andrestc.com/post/cgroups-io/
https://facebookmicrosites.github.io/cgroup2/docs/create-cgroups.html
    $ stat -fc %T /sys/fs/cgroup/
    # For cgroup v2, the output is cgroup2fs
    # For cgroup v1, the output is tmpfs.

???
https://github.com/mk-fg/systemd-cgroup-nftables-policy-manager
    ...
    Both nftables/iptables rules use xt_cgroup kernel module that - when looking at the packet - actually matches numeric cgroup ID, and not the path string, 
    and does not update those IDs dynamically when cgroups are created/removed in any way.
xt_cgroup patchset
https://lwn.net/Articles/667117/