trending/OpenFlux
1
0
Fork 0
mirror of https://github.com/p1neappleXpress/OpenFlux.git synced 2026-10-02 05:04:39 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge p1neappleXpress/main into node-transports

Browse Source 
Takes node-v1.1.0 (main's release) into node-install.sh, keeps the
script's RELEASE_REPO/GITHUB_* layout. The wizard's links go through
share.MakeLink and provision.SessionContext through transport.KDFContexts,
the core's one context rule, instead of rules of their own;
TestNodeConfMatchesShareLink checks node.conf against it.
This commit is contained in:
meepo161
2026-09-28 23:29:56 +03:00
parent e193bd9a24 2ec01a5bf0
commit fe9dc8b0fc
53 changed files with 4373 additions and 1013 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/node-release.yml
+7 -1
View File
@@ -48,6 +48,12 @@ jobs:
env:
GH_TOKEN: ${{ github.token }}
run: |
# What the node runs: the newest released section of CHANGELOG.md.
{
echo "Ядро выходной ноды для мастера «Своя нода» в OpenFlux (десктоп и Android). Хеши сверены с deploy/node-install.sh."
echo
awk 'index($0, "## [")==1 { if (f) exit; if (index($0, "Unreleased") == 0) { f = 1; print "## Ядро " substr($0, 4) }; next } f' CHANGELOG.md
} > notes.md
gh release create "$GITHUB_REF_NAME" dist/openflux-linux-* dist/SHA256SUMS \
--title "OpenFlux node $GITHUB_REF_NAME" \
--notes "Ядро выходной ноды для мастера «Своя нода» в OpenFlux (десктоп и Android). Хеши сверены с deploy/node-install.sh."
--notes-file notes.md
.github/workflows/release.yml
+4 -1
View File
@@ -77,7 +77,10 @@ jobs:
run: |
if [[ "$GITHUB_REF" == refs/tags/v* ]]; then tag="${GITHUB_REF#refs/tags/}"
else tag="v${{ inputs.version }}"; fi
# The release shows its CHANGELOG.md section, then what the assets are.
awk -v v="${tag#v}" 'index($0, "## [")==1 { if (f) exit; f = (index($0, "## [" v "]") == 1); next } f' CHANGELOG.md > notes.md
printf '\n%s\n' "Кросс-платформенная сборка CLI (клиент/выходная нода): \`openflux-<os>-<arch>[.exe]\`. Контрольные суммы — в \`SHA256SUMS.txt\`. Для установки выходной ноды по SSH из приложения используется отдельный релиз \`node-v*\` (см. deploy/node-install.sh)." >> notes.md
gh release create "$tag" dist/* \
--title "OpenFlux $tag" \
--generate-notes \
--notes "Кросс-платформенная сборка CLI (клиент/выходная нода): \`openflux-<os>-<arch>[.exe]\`. Контрольные суммы — в \`SHA256SUMS.txt\`. Для установки выходной ноды по SSH из приложения используется отдельный релиз \`node-v*\` (см. deploy/node-install.sh)."
--notes-file notes.md
CHANGELOG.md
+62
View File
@@ -19,6 +19,68 @@ All notable changes to the OpenFlux core. Format loosely follows
channel does not stay up. An app with an older pinned script no longer
downgrades a server the updater has moved on.
## [0.2.0] - 2026-09-28
Every client now behaves alike: peers of different builds and modes find
each other instead of dropping every packet in silence. The node wizard
(desktop and Android) installs this core as `node-v1.1.0`.
### Added
- Classic compatibility inside the Session (`PROTOCOL_NEGOTIATION.md`):
a classic setup with a key (`--transport=X`, the apps' classic profiles)
runs a Session and speaks classic to an exit that does not answer the
handshake, switching once it does; a classic-configured exit serves
classic and Session clients. `--negotiate` stays strict; exits configured
as a Session (wizard, `.conf`, `--transports`) serve Session clients only.
- Codec fallback: the classic codec decodes batch-v2 and legacy frames,
sends what the peer sends, and the client tries the other framing when
the peer is silent. `--codec` is a preference now, not a requirement.
- KDF context fallback: one rule for the context (`transport.KDFContexts`)
and alternates for what other builds derive; a record that fails under
the current keys is tried under them, the exit answers under the
client's context, a silent client cycles through them.
- `mobile/ios`: the iOS C library (`build_ios.sh`) on package `mobile`,
replacing the root `export_ios*.go`: the calls the iOS app makes, plus
Session profiles (`OpenFluxShareDecode` returns one ready to start),
mode and captcha calls. An app that links this core as a submodule gets
the same Session, links and fallbacks as Android.
- `mobile`: `ShareSessionSpecs`, `SetInitialCookies`, `SetLowMemory`
(Volga's new `SlimVolgaConfig` for the iOS extension), `ReadTimeout`,
`ConnectionMode`.
- Links are read and made by the core only. `share.Read` / `share.Make`
answer every entry point with the same JSON (`--parse-link`, new
`--make-link`, `mobile.ReadShareLink` / `MakeShareLink`, the iOS
`OpenFluxShareDecode` / `OpenFluxShareEncode`): the configuration and
its context, the link, or an error `code` (and `param`) that the apps
put in their own words. `share.Make` normalizes what apps spell
differently (the default codec is left out, an encrypted link always
names its context, by the one rule when not given), so one
configuration gives one link on every client; the node wizard's link,
desktop and Android, comes from one `share.NodeConfig`.
- Logs a user can act on without `-dd`: key or context mismatch, the peer
running the other layering, codec and context fallbacks, a second client
taking over the exit, carriers failing to start, documents dropping, and
a diagnosis when the handshake does not complete.
### Fixed
- A classic cupsonline client given the rooms as `--url` derived another
key than the exit that created them: nothing got through.
- boards sent engine.io pings from the client, which an EIO=4 server
answers by closing the socket: the board dropped every 20 seconds.
- A Session client's cupsonline carrier was built as an exit: with no or
dead rooms it created rooms of its own and waited in them.
- openflux:// links: base64 padding, the standard alphabet, whitespace and
line breaks are accepted; secrets are counted in characters as Kotlin
counts them, not bytes.
- Carrier names that differ between the two sides no longer break cookie
exchange and exit checks (messages carry the document URL).
- yandex / mailru: a socket whose keepalive failed is closed so the
reconnect runs; writes are bounded.
- `utils.Infof` reaches the apps' log screens.
- A Session stopped each carrier twice.
## [0.1.0] - 2026-09-27
First release from the current `main` line (encrypted-logging + the
PROTOCOL_NEGOTIATION.md
+97 -11
View File
@@ -1,16 +1,40 @@
# Authenticated negotiation v1
Opt-in CLI extension to the existing encrypted packet stack, not a new cipher
suite or a production security certification. Both peers need `--negotiate`,
batched codec and the same encryption secret/context. Old clients require the
unchanged default mode on the exit. No automatic downgrade is implemented.
Not a new cipher suite or a production security certification: an
authenticated session (the Session) on top of the existing encrypted packet
stack. Both peers need the same encryption secret and context (see
"Encryption context"). A Session runs whenever a peer is configured with a
key: `--negotiate`, `--transports`, `.conf` transports, the apps' Session
profiles, and also classic setups (`--transport=X` with a key, the apps'
classic profiles), which keep classic compatibility (see "Classic
compatibility").
## Layering on a carrier
There are two layerings, and implementations must match the one they
speak byte for byte:
Session IPv4 -> envelope -> batch-v2 frame of envelopes -> AES-GCM record -> carrier
classic IPv4 -> AES-GCM record -> codec frame (batch-v2 or legacy) -> carrier
In the Session the batch frame (zstd when it helps) is encrypted as a whole,
so every carrier frame starts with the record header `OFX` (0x4F 0x46 0x58),
version 1 and the direction byte. In the classic layering each IPv4 packet is
encrypted on its own and the records are framed by the codec, so a carrier
frame starts with 0x02 (batch-v2) or 0x00 / 0x1F (legacy per-packet, raw /
LZ4). Receive reverses the order. Because the first byte differs, one
carrier can carry both, and a receiver can tell which layering its peer runs.
(An earlier version of this document gave the Session's order as
envelope -> AES-GCM -> batch-v2, which is the classic order; the code has
always encrypted the batch frame. A Session built from that description
does not interoperate.)
## Envelope
Send order: IPv4 -> negotiation envelope -> existing AES-GCM packet -> batch-v2
framing/compression -> carrier. Receive reverses that order. The envelope is
inside AEAD, including its type, role, identities, capabilities and sequence.
Integers are unsigned big-endian; unknown versions/types/reserved bits fail closed.
The envelope is inside AEAD, including its type, role, identities,
capabilities and sequence. Integers are unsigned big-endian; unknown
versions/types/reserved bits fail closed.
| Bytes | Meaning |
| --- | --- |
@@ -38,8 +62,10 @@ Readiness requires a valid peer hello echoing the current local challenge.
The effective policy is the capability intersection and smaller packet limit.
The established peer cannot change its policy through later hellos. Retries
with an unconfirmed ready bit receive a fresh confirmation, including after the
other side has completed Start. The client's handshake deadline is 20 seconds;
the exit never initiates and waits for a client indefinitely.
other side has completed Start. The client's handshake deadline is 20 seconds
(a client with classic compatibility starts sending classic after 3 and
keeps offering the handshake); the exit never initiates and waits for a
client indefinitely.
A hello from a different sender while established usually means the peer
restarted, but may be old traffic replayed from a carrier (anyone with access
@@ -91,7 +117,11 @@ are hashed across carriers only when they share that priority.
| 0x20 LinkPing, 0x21 LinkPong | both | none |
An empty `transport` (older peers) means the highest-priority transport that
carries cookies. Unknown subtypes are passed to the application and otherwise
carries cookies. Cookie messages and AuthRequired may carry `"doc"`, the
document URL of that transport on the sender's side: the two sides do not
always name carriers alike, so a receiver that has no carrier of that name
matches by `doc`, then by the type the name starts with when it has one
carrier of that type. Unknown subtypes are passed to the application and otherwise
ignored.
## Checks on the exit
@@ -112,6 +142,57 @@ below gVisor's ephemeral range and common OS ones; replies to those ports go to
it, everything else to the regular client path. The proxy listens on loopback
without authentication while it runs, like the SOCKS5 inbound.
## Classic compatibility
A peer configured classic with a key runs a Session with the classic
layering next to it on its one carrier:
- A client falls back: it offers the handshake and, until the exit answers
(it waits 3 seconds at start), sends IPv4 in the classic layering, so an
exit that predates Session or runs without it still works. It keeps
offering the handshake (every 2 seconds after the first 20, every 10 once
the exit answered classic) and switches to the Session when answered. A
single-carrier Session client (a Session profile or `--transports` with
one carrier) falls back the same way; `--negotiate` does not.
- An exit configured classic serves classic clients as well as Session
ones, as long as no Session client was heard within the link timeout:
classic frames that arrive while a Session client is active are dropped
(a replayed capture cannot take the replies away from it).
- An exit configured as a Session (`--negotiate`, `--transports`, `.conf`
transports, the node wizard, the apps' Session exits) serves Session
clients only; classic frames are dropped and logged with the fix.
## Codec (classic layering)
Both framings are decoded whichever the peer uses. A peer sends batch-v2
once it has received a batch frame (an empty one, `02 00`, is a capability
probe), legacy while it has received only legacy frames, and its preferred
framing (`--codec`) before it has heard anything. The side that speaks
first (the client) switches to the other framing after 2 seconds without
an answer, then every second, so a peer that decodes one framing only is
still reached. A lone 0x00 is a carrier keepalive (Volga), not a frame.
## Encryption context
The scrypt salt is `SHA-256("OpenFlux encrypted transport v1\0" + context)`.
Every peer picks the context with the same rule (`transport.KDFContexts`):
1. an explicit context (`--session-context`, the context an openflux://
link carries);
2. `--url`, unless it is a cupsonline room list;
3. the URL of the highest-priority carrier that names the channel
(not cupsonline: its exit creates the room list at start; not direct:
host:port differs between the sides; not oneme);
4. `http://#`.
Builds have derived it differently (the classic cupsonline client used the
room list; panels and an older fork used the transport name), so each peer
also knows the alternates other builds derive for its setup. A record that
does not open under the current keys is tried under them; the exit answers
under the context the client used, and a client that hears nothing moves to
the next candidate after 4 seconds, then every 2. The context is a public
salt: accepting several weakens nothing, each one still requires the secret.
## Limits and compatibility
Shared-secret holders are trusted peers. The existing static key derivation is
@@ -119,6 +200,11 @@ unchanged: no forward secrecy, automatic key rotation or protection after secret
compromise is claimed. AEAD authenticates packets; capability assertions still
describe configured software functionality, not a live Internet reachability test.
The classic layering has no challenge binding, sequence window or
capability negotiation, only AES-GCM with a bounded nonce replay cache.
A client that falls back to it can be kept there by whoever drops the
exit's handshake answers; `--negotiate` rules that out on both sides.
ICMP-error support refers to errors returned from a raw exit to the client; it
does not promise bidirectional arbitrary ICMP, IPv6, echo or redirects. There
is no active path-MTU probing. The separate raw-exit ICMP/NAT implementation
README.md
+4 -2
View File
@@ -221,8 +221,6 @@ OpenFlux/
tun_watch.go # Socket watcher for bypass routes
tun_learn.go, tun_other.go # utun helpers / non-darwin stubs
signals_{unix,windows}.go # Shutdown signals
export_ios.go # cgo bridge for the iOS static library
export_ios_packet.go # iOS packet tunnel bridge
transport/
transport.go # Transport interface
batched.go # BatchedTransport (coalescing + zstd)
@@ -265,6 +263,8 @@ OpenFlux/
network/ # Checksums, packet parsing
utils/ # Logging
ios-app/ # SwiftUI iOS client (XcodeGen)
mobile/ # App bridge: gomobile (Android) and the iOS
# C library (mobile/ios, liboflux.a)
build_all.sh # Cross-build release binaries
build_ios.sh # Build iOS static library (liboflux.a)
build_ios_app.sh # Build + archive + export iOS app IPA
@@ -613,6 +613,8 @@ Measure raw goodput through the transport, without touching the host network:
| `--share` | | `false` | Exit: print an `openflux://` link and QR code for clients |
| `--share-host` | | (first public IPv4) | Exit: address clients dial for `direct` in that link |
| `--node-wizard` | | | Sole argument: run the JSON-over-stdio provisioning protocol instead of normal CLI startup (see [Highlights](#highlights)) |
| `--parse-link` | | | `--parse-link <link\|->`: read an openflux:// link (`-`: from stdin) and print `{"config","context"}` or `{"error","code","param"}` as JSON; the reading every client uses |
| `--make-link` | | | `--make-link <json\|->`: build the link for a share configuration (`-`: from stdin) and print `{"link","config","context"}` or the error, as every client exports it |
Deprecated (kept for one release, mapped automatically to the new flags):
`--client`, `--exit-node`, `--tun`, `--socks5-mode`, `--legacy`,
README.ru.md
+4 -3
View File
@@ -228,8 +228,6 @@ OpenFlux/
tun_watch.go # Watcher сокетов для bypass-маршрутов
tun_learn.go, tun_other.go # Хелперы utun / заглушки для не-darwin
signals_{unix,windows}.go # Сигналы завершения
export_ios.go # cgo-мост для iOS-статической библиотеки
export_ios_packet.go # Мост packet tunnel для iOS
transport/
transport.go # Интерфейс Transport
batched.go # BatchedTransport (склейка + zstd)
@@ -272,7 +270,8 @@ OpenFlux/
network/ # Контрольные суммы, разбор пакетов
utils/ # Логирование
ios-app/ # iOS-клиент на SwiftUI (XcodeGen)
mobile/ # gomobile-мост для Android/iOS (см. ниже)
mobile/ # Мост для приложений: gomobile (Android) и
# C-библиотека iOS (mobile/ios, liboflux.a)
build_all.sh # Кросс-сборка релизных бинарников
build_ios.sh # Сборка статической библиотеки iOS (liboflux.a)
build_ios_app.sh # Сборка + архив + экспорт IPA iOS
@@ -605,6 +604,8 @@ URL = YOUR_YANDEX_DOC_URL
| `--share` | | `false` | Выходная нода: напечатать ссылку `openflux://` и QR-код для клиентов |
| `--share-host` | | (первый публичный IPv4) | Выходная нода: адрес для `direct` в этой ссылке |
| `--node-wizard` | | | Единственный аргумент: запускает протокол развёртывания по JSON-over-stdio вместо обычного запуска CLI (см. [Ключевые особенности](#ключевые-особенности)) |
| `--parse-link` | | | `--parse-link <ссылка\|->`: читает ссылку openflux:// (`-`: из stdin) и печатает JSON `{"config","context"}` или `{"error","code","param"}` — тот же разбор, что у всех клиентов |
| `--make-link` | | | `--make-link <json\|->`: собирает ссылку из конфигурации (`-`: из stdin) и печатает `{"link","config","context"}` или ошибку — так ссылку выпускает любой клиент |
Устаревшие (оставлены на один релиз, автоматически маппятся на новые флаги):
`--client`, `--exit-node`, `--tun`, `--socks5-mode`, `--legacy`,
build_ios.sh
+14 -6
View File
@@ -3,6 +3,10 @@ set -e
OUTPUT_DIR="output/ios"
LIBRARY_NAME="liboflux"
# Paths are this checkout's, whatever the caller's directory: an app that
# links the core as a submodule runs core/build_ios.sh from its own root.
ROOT="$(cd "$(dirname "$0")" && pwd)"
OUT="$ROOT/$OUTPUT_DIR/$LIBRARY_NAME.a"
# Paths configuration
XCODE_PATH="${XCODE_PATH:-/Applications/Xcode.app}"
@@ -10,7 +14,7 @@ DEVELOPER_DIR="$XCODE_PATH/Contents/Developer"
SDK_PATH="$DEVELOPER_DIR/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk"
CLANG="$DEVELOPER_DIR/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang"
mkdir -p "$OUTPUT_DIR"
mkdir -p "$ROOT/$OUTPUT_DIR"
# Verify paths
if [ ! -d "$SDK_PATH" ]; then
@@ -36,17 +40,21 @@ export CGO_LDFLAGS="-isysroot $SDK_PATH -arch arm64 -miphoneos-version-min=13.0"
echo "Building for iOS (arm64)..."
# Build static library
if go build \
# The iOS C API lives in mobile/ios, on package mobile: the same Session,
# context rule, codec and link handling as the Android library. An app
# that links this checkout as a submodule gets exactly this core.
if (cd "$ROOT/mobile" && go build \
-buildmode=c-archive \
-tags ios \
-ldflags="-w" \
-trimpath \
-o "$OUTPUT_DIR/$LIBRARY_NAME.a" \
. ; then
-o "$OUT" \
./ios) ; then
# Header liboflux.h is generated automatically by cgo from //export directives.
echo "Build complete: $OUTPUT_DIR/$LIBRARY_NAME.a"
ls -lh "$OUTPUT_DIR/$LIBRARY_NAME.a"
echo "Build complete: $OUT"
ls -lh "$OUT"
else
echo "Build failed"
deploy/node-install.sh
+4 -4
View File
@@ -46,10 +46,10 @@
set -u
umask 077
CORE_VERSION="node-v1.0.1"
SHA_amd64="9fa157550d2c20c0bc03c12823b4ad0140ba070199b5548eacf98c5a2cca6cb8"
SHA_arm64="325335fa416d2f87cba84c5a85d865c596169cd79c7f4cfc916cd67a88612886"
SHA_arm="7f280b01a53bee33e84a7525e035f1e09f51e9070b612b903e492c45c6edf300"
CORE_VERSION="node-v1.1.0"
SHA_amd64="9ec36c073749c1d02ca163516ba6fc257cc624a69833fb108de65ba4400e8f41"
SHA_arm64="b6d74ae230d9f4711cc4e03ba19d9eb7b4e3987ae6eeb45f86257743da9623ed"
SHA_arm="77c5afa26566db77b465e26bc0bdcde55e9f2babb08b386953a3d2f769667cf8"
# The repository this script and its core come from: the core is one of its
# node-v* releases, and the updater follows them (UPDATE_CONF may override
# that with a "repo=owner/name" line).
export_ios.go
-296
View File
@@ -1,296 +0,0 @@
//go:build ios
package main
/*
#include <stdlib.h>
*/
import "C"
import (
"context"
"crypto/tls"
"fmt"
"net"
"strconv"
"strings"
"sync"
"time"
"unsafe"
"openflux/socks5"
"openflux/transport"
"openflux/transport/oneme"
"openflux/transport/yandex"
"openflux/tunnel"
"openflux/utils"
)
// ---- log ring buffer piped into the app UI ----
type ringLog struct {
mu sync.Mutex
lines []string
}
func (r *ringLog) Write(p []byte) (int, error) {
r.mu.Lock()
defer r.mu.Unlock()
r.lines = append(r.lines, strings.TrimRight(string(p), "\n"))
if len(r.lines) > 1000 {
r.lines = r.lines[len(r.lines)-1000:]
}
return len(p), nil
}
func (r *ringLog) drain() string {
r.mu.Lock()
defer r.mu.Unlock()
if len(r.lines) == 0 {
return ""
}
out := strings.Join(r.lines, "\n")
r.lines = r.lines[:0]
return out
}
var logbuf = &ringLog{}
// ---- running client state ----
var (
stateMu sync.Mutex
running bool
socks *socks5.SOCKS5Server
trans transport.Transport
)
func init() {
// Route log output into the ring buffer, but leave verbose logging OFF by
// default (production). The app can turn it on via OpenFluxSetDebug; the
// per-packet logging is expensive.
utils.SetOutput(logbuf)
// The client's local (mobile) DNS may be poisoned for censored hosts
// (observed: ifconfig.me -> 240.0.1.72, a reserved address). Resolve names
// over DNS-over-TLS instead so DialTCP gets real IPs to hand the exit node.
net.DefaultResolver = &net.Resolver{
PreferGo: true,
StrictErrors: false,
Dial: dialSecureDNS,
}
}
// dotServer is a DNS-over-TLS endpoint (addr:853 + TLS SNI).
type dotServer struct {
addr string
sni string
}
var dotServers = []dotServer{
{"77.88.8.8:853", "common.dot.dns.yandex.net"}, // Yandex, reachable in-region
{"8.8.8.8:853", "dns.google"},
{"1.1.1.1:853", "cloudflare-dns.com"},
}
// dialSecureDNS opens a DNS-over-TLS connection for net.Resolver, trying the
// configured servers in order.
func dialSecureDNS(ctx context.Context, _, _ string) (net.Conn, error) {
var lastErr error
for _, s := range dotServers {
d := tls.Dialer{
NetDialer: &net.Dialer{Timeout: 6 * time.Second},
Config: &tls.Config{ServerName: s.sni, MinVersion: tls.VersionTLS12},
}
conn, err := d.DialContext(ctx, "tcp", s.addr)
if err == nil {
return conn, nil
}
lastErr = err
utils.Debugf("[DNS] DoT %s failed: %v", s.addr, err)
}
return nil, lastErr
}
// Return codes for OpenFluxStartClient.
const (
startOK = 0
startAlreadyRunning = 1
startBadTransport = 2
startTransportError = 3
startAddrInUse = 4 // SOCKS5 port could not be bound (e.g. already in use)
startPanic = 5
)
// OpenFluxStartClient starts the SOCKS5 client tunnel.
//
// transportType: "yandex" or "oneme".
// url: Yandex.Docs document URL (yandex transport).
// socksAddr: e.g. "127.0.0.1:1080".
// maxToken/maxUid: credentials for the "oneme" (MAX) transport; pass "" for yandex.
//
// Returns 0 on success, non-zero on error (see start* codes; details go to the log).
//
//export OpenFluxStartClient
func OpenFluxStartClient(transportType, url, socksAddr, maxToken, maxUid *C.char) (rc C.int) {
tt := C.GoString(transportType)
docURL := C.GoString(url)
addr := C.GoString(socksAddr)
mToken := C.GoString(maxToken)
mUid := C.GoString(maxUid)
// Never let a panic unwind into the C/Swift caller and crash the app.
defer func() {
if r := recover(); r != nil {
utils.Debugf("[BRIDGE] Recovered from panic in start: %v", r)
rc = C.int(startPanic)
}
}()
stateMu.Lock()
defer stateMu.Unlock()
if running {
utils.Debugf("[BRIDGE] Start ignored: already running")
return C.int(startAlreadyRunning)
}
// Bind the SOCKS5 port up front so "address already in use" is reported
// cleanly to the UI instead of failing later in a background goroutine.
probe, err := net.Listen("tcp", addr)
if err != nil {
utils.Debugf("[BRIDGE] Cannot bind %s: %v", addr, err)
return C.int(startAddrInUse)
}
probe.Close()
config := transport.DefaultConfig()
var t transport.Transport
switch tt {
case "yandex", "":
t = transport.NewCompressedTransport(yandex.NewYandexDocsTransport(docURL, config))
case "oneme":
uidint, _ := strconv.ParseInt(mUid, 10, 64)
t = transport.NewCompressedTransport(oneme.NewOneMeTransport(false, mToken, uidint, config))
default:
utils.Debugf("[BRIDGE] Unknown transport type: %s", tt)
return C.int(startBadTransport)
}
if err := t.Start(); err != nil {
utils.Debugf("[BRIDGE] Failed to start transport: %v", err)
return C.int(startTransportError)
}
tun := tunnel.NewTCPTunnel(t, false)
srv := socks5.NewSOCKS5Server(addr, tun)
if err := srv.Bind(); err != nil {
utils.Debugf("[BRIDGE] Cannot bind %s: %v", addr, err)
t.Stop()
return C.int(startAddrInUse)
}
trans = t
socks = srv
running = true
go func() {
defer func() {
if r := recover(); r != nil {
utils.Debugf("[BRIDGE] Recovered from panic in SOCKS5 loop: %v", r)
}
}()
utils.Debugf("[BRIDGE] Client running (SOCKS5 on %s, transport %s)", addr, tt)
if err := srv.Start(); err != nil {
utils.Debugf("[BRIDGE] SOCKS5 server stopped: %v", err)
}
}()
return C.int(startOK)
}
// OpenFluxStop stops the running client (transport + SOCKS5 listener).
//
//export OpenFluxStop
func OpenFluxStop() {
stateMu.Lock()
defer stateMu.Unlock()
if !running {
return
}
if socks != nil {
socks.Close()
}
if trans != nil {
trans.Stop()
}
socks = nil
trans = nil
running = false
utils.Debugf("[BRIDGE] Stopped")
}
// OpenFluxIsRunning returns 1 if the client is running, 0 otherwise.
//
//export OpenFluxIsRunning
func OpenFluxIsRunning() C.int {
stateMu.Lock()
defer stateMu.Unlock()
if running {
return C.int(1)
}
return C.int(0)
}
// OpenFluxIsConnected returns 1 if the transport reports a live connection.
//
//export OpenFluxIsConnected
func OpenFluxIsConnected() C.int {
stateMu.Lock()
defer stateMu.Unlock()
if trans != nil && trans.IsConnected() {
return C.int(1)
}
return C.int(0)
}
// OpenFluxStatsJSON returns a small JSON blob with transport stats.
// The returned string is C-allocated; free it with OpenFluxFreeString.
//
//export OpenFluxStatsJSON
func OpenFluxStatsJSON() *C.char {
stateMu.Lock()
defer stateMu.Unlock()
if trans == nil {
return C.CString(`{"running":false}`)
}
s := trans.Stats()
js := fmt.Sprintf(
`{"running":%t,"connected":%t,"bytesSent":%d,"bytesReceived":%d,"packetsSent":%d,"packetsRecv":%d,"reconnects":%d,"uptimeSec":%d}`,
running, s.Connected, s.BytesSent, s.BytesReceived, s.PacketsSent, s.PacketsRecv, s.Reconnects,
int64(s.Uptime/time.Second),
)
return C.CString(js)
}
// OpenFluxReadLog drains buffered log lines (newline-separated).
// The returned string is C-allocated; free it with OpenFluxFreeString.
//
//export OpenFluxReadLog
func OpenFluxReadLog() *C.char {
return C.CString(logbuf.drain())
}
// OpenFluxFreeString frees a string returned by this library.
//
//export OpenFluxFreeString
func OpenFluxFreeString(s *C.char) {
C.free(unsafe.Pointer(s))
}
// OpenFluxSetDebug toggles verbose (per-packet) logging at runtime. Off by
// default; enabling it costs CPU, so only turn it on while debugging.
//
//export OpenFluxSetDebug
func OpenFluxSetDebug(on C.int) {
utils.SetDebug(on != 0)
}
export_ios_packet.go
-332
View File
@@ -1,332 +0,0 @@
//go:build ios
package main
/*
#include <stdlib.h>
*/
import "C"
import (
"context"
"crypto/tls"
"encoding/binary"
"io"
"net"
"runtime/debug"
"strconv"
"sync"
"sync/atomic"
"time"
"unsafe"
"openflux/network"
"openflux/transport"
"openflux/transport/oneme"
"openflux/transport/yandex"
"openflux/utils"
)
// Packet-tunnel (NEPacketTunnelProvider) mode — pure L3 forwarding.
//
// The device is given tunnel address 10.10.10.2, which is exactly what the exit
// node expects (it hardcodes returns to 10.10.10.2). So we forward the device's
// raw IPv4 TCP and UDP packets straight over the transport — no gVisor stack on
// the client, which keeps the extension well under its memory cap. UDP DNS
// retains the existing local DNS-over-TLS path for compatibility with older
// TCP-only exits. Other UDP requires explicit opt-in for a UDP-capable exit.
//
// Uses startOK / start* codes and dotServers from export_ios.go.
const tunClientIP = "10.10.10.2"
var (
ptMu sync.Mutex
ptOn bool
ptTrans transport.Transport
ptOutQ chan []byte
ptCtx context.Context
ptCancel context.CancelFunc
)
//export OpenFluxStartPacketTunnel
func OpenFluxStartPacketTunnel(transportType, url, maxToken, maxUid *C.char) (rc C.int) {
tt := C.GoString(transportType)
docURL := C.GoString(url)
mToken := C.GoString(maxToken)
mUid := C.GoString(maxUid)
defer func() {
if r := recover(); r != nil {
utils.Debugf("[PKT] Recovered from panic in start: %v", r)
rc = C.int(startPanic)
}
}()
ptMu.Lock()
defer ptMu.Unlock()
if ptOn {
return C.int(startAlreadyRunning)
}
// Keep the extension well under the NE memory cap.
debug.SetMemoryLimit(40 << 20)
debug.SetGCPercent(20)
config := transport.DefaultConfig()
var t transport.Transport
switch tt {
case "yandex", "":
t = transport.NewCompressedTransport(yandex.NewYandexDocsTransport(docURL, config))
case "oneme":
uidint, _ := strconv.ParseInt(mUid, 10, 64)
t = transport.NewCompressedTransport(oneme.NewOneMeTransport(false, mToken, uidint, config))
default:
return C.int(startBadTransport)
}
outQ := make(chan []byte, 1024)
// Packets coming back from the exit node -> queue for the device.
t.Receive(func(data []byte) {
network.LogPacket("PKT", network.DirInbound, data)
select {
case outQ <- append([]byte(nil), data...):
default: // queue full: drop, TCP will retransmit
}
})
if err := t.Start(); err != nil {
utils.Debugf("[PKT] transport start failed: %v", err)
return C.int(startTransportError)
}
ptTrans = t
ptOutQ = outQ
ptCtx, ptCancel = context.WithCancel(context.Background())
ptOn = true
utils.Debugf("[PKT] L3 packet tunnel started (transport %s)", tt)
return C.int(startOK)
}
var ptUDPEnabled atomic.Bool
// OpenFluxTunSetUDPEnabled enables non-DNS UDP for a known UDP-capable exit.
// Disabled by default: legacy exits otherwise silently blackhole QUIC traffic.
//
//export OpenFluxTunSetUDPEnabled
func OpenFluxTunSetUDPEnabled(enabled C.int) { ptUDPEnabled.Store(enabled != 0) }
// OpenFluxTunWritePacket forwards one device IPv4 TCP or UDP packet.
//
//export OpenFluxTunWritePacket
func OpenFluxTunWritePacket(buf *C.char, length C.int) {
defer func() { _ = recover() }() // never let a bad packet crash the extension
if buf == nil || length < 20 || length > 65535 {
return
}
ptMu.Lock()
t := ptTrans
outQ := ptOutQ
ptMu.Unlock()
if t == nil {
return
}
pkt := C.GoBytes(unsafe.Pointer(buf), length)
if pkt[0]>>4 != 4 { // IPv4 only
return
}
ihl := int(pkt[0]&0x0f) * 4
total := int(binary.BigEndian.Uint16(pkt[2:4]))
if ihl < 20 || total < ihl || total > len(pkt) {
return
}
pkt = pkt[:total]
network.LogPacket("PKT", network.DirOutbound, pkt)
switch pkt[9] { // protocol
case 6: // TCP
t.Send(pkt)
case 17: // UDP
if len(pkt) < ihl+8 || binary.BigEndian.Uint16(pkt[6:8])&0x3fff != 0 {
return
}
udpLen := int(binary.BigEndian.Uint16(pkt[ihl+4 : ihl+6]))
if udpLen < 8 || udpLen > len(pkt)-ihl {
return
}
dstPort := binary.BigEndian.Uint16(pkt[ihl+2 : ihl+4])
if dstPort != 53 {
if ptUDPEnabled.Load() {
_ = t.Send(pkt)
} else {
sendICMPPortUnreachable(pkt, outQ)
}
return
}
pkt = pkt[:ihl+udpLen]
select {
case dnsSem <- struct{}{}:
go func() { defer func() { <-dnsSem }(); handleDNSPacket(pkt, outQ) }()
default:
}
}
}
var dnsSem = make(chan struct{}, 16)
func sendICMPPortUnreachable(orig []byte, outQ chan []byte) {
ihl := int(orig[0]&0x0f) * 4
quote := orig[:ihl+8]
icmp := make([]byte, 8+len(quote))
icmp[0], icmp[1] = 3, 3
copy(icmp[8:], quote)
binary.BigEndian.PutUint16(icmp[2:4], network.IPChecksum(icmp))
pkt := make([]byte, 20+len(icmp))
pkt[0], pkt[8], pkt[9] = 0x45, 64, 1
binary.BigEndian.PutUint16(pkt[2:4], uint16(len(pkt)))
copy(pkt[12:16], orig[16:20])
copy(pkt[16:20], orig[12:16])
binary.BigEndian.PutUint16(pkt[10:12], network.IPChecksum(pkt[:20]))
copy(pkt[20:], icmp)
select {
case outQ <- pkt:
default:
}
}
// OpenFluxTunReadPacket blocks for the next packet destined to the device.
//
//export OpenFluxTunReadPacket
func OpenFluxTunReadPacket(buf *C.char, max C.int) C.int {
if buf == nil || max <= 0 {
return 0
}
ptMu.Lock()
outQ := ptOutQ
ctx := ptCtx
ptMu.Unlock()
if outQ == nil || ctx == nil {
return 0
}
select {
case data := <-outQ:
n := len(data)
if n > int(max) {
n = int(max)
}
dst := unsafe.Slice((*byte)(unsafe.Pointer(buf)), int(max))
copy(dst[:n], data[:n])
return C.int(n)
case <-ctx.Done():
return 0
}
}
//export OpenFluxStopPacketTunnel
func OpenFluxStopPacketTunnel() {
ptMu.Lock()
defer ptMu.Unlock()
if !ptOn {
return
}
if ptCancel != nil {
ptCancel()
}
if ptTrans != nil {
ptTrans.Stop()
}
ptTrans = nil
ptOutQ = nil
ptOn = false
utils.Debugf("[PKT] L3 packet tunnel stopped")
}
// handleDNSPacket answers a device DNS query over DNS-over-TLS and enqueues a
// UDP response packet back to the device.
func handleDNSPacket(req []byte, outQ chan []byte) {
defer func() { _ = recover() }()
ihl := int(req[0]&0x0f) * 4
if len(req) < ihl+8 {
return
}
srcIP := req[12:16]
dstIP := req[16:20]
srcPort := req[ihl : ihl+2]
dstPort := req[ihl+2 : ihl+4]
query := req[ihl+8:]
if len(query) == 0 {
return
}
answer, err := dnsOverTLS(query)
if err != nil || len(answer) == 0 {
utils.Debugf("[DNS] resolve failed: %v", err)
return
}
udpLen := 8 + len(answer)
if udpLen > 65535-20 {
return
}
// Build a fresh minimal IPv4 header; do not advertise uncopied options.
ihl = 20
total := ihl + udpLen
resp := make([]byte, total)
resp[0] = 0x45
resp[1] = req[1]
binary.BigEndian.PutUint16(resp[2:4], uint16(total))
resp[8] = 64
resp[9] = 17
copy(resp[12:16], dstIP)
copy(resp[16:20], srcIP)
ipck := network.IPChecksum(resp[:20])
resp[10] = byte(ipck >> 8)
resp[11] = byte(ipck)
copy(resp[ihl:ihl+2], dstPort)
copy(resp[ihl+2:ihl+4], srcPort)
binary.BigEndian.PutUint16(resp[ihl+4:ihl+6], uint16(udpLen))
copy(resp[ihl+8:], answer)
select {
case outQ <- resp:
default:
}
}
func dnsOverTLS(query []byte) ([]byte, error) {
var lastErr error
for _, s := range dotServers {
answer, err := dotQueryOne(s, query)
if err == nil {
return answer, nil
}
lastErr = err
}
return nil, lastErr
}
func dotQueryOne(s dotServer, query []byte) ([]byte, error) {
dialer := tls.Dialer{
NetDialer: &net.Dialer{Timeout: 6 * time.Second},
Config: &tls.Config{ServerName: s.sni, MinVersion: tls.VersionTLS12},
}
conn, err := dialer.DialContext(context.Background(), "tcp", s.addr)
if err != nil {
return nil, err
}
defer conn.Close()
conn.SetDeadline(time.Now().Add(6 * time.Second))
var length [2]byte
binary.BigEndian.PutUint16(length[:], uint16(len(query)))
if _, err := conn.Write(append(length[:], query...)); err != nil {
return nil, err
}
if _, err := io.ReadFull(conn, length[:]); err != nil {
return nil, err
}
answer := make([]byte, binary.BigEndian.Uint16(length[:]))
if _, err := io.ReadFull(conn, answer); err != nil {
return nil, err
}
return answer, nil
}
ios-app/README.md
+36 -4
View File
@@ -10,10 +10,42 @@ SOCKS5 tunnel over the Yandex.Docs transport on `127.0.0.1:1080`.
- `ExportOptions.plist` — App Store export options (team 8GQH8GQ252, automatic signing).
## Go bridge API (liboflux.h)
- `OpenFluxStartClient(transportType, url, socksAddr, maxToken, maxUid)` — start the client (returns 0 on success).
- `OpenFluxStop()` — stop transport + SOCKS5 listener.
- `OpenFluxIsRunning()` / `OpenFluxIsConnected()` — state.
- `OpenFluxStatsJSON()` / `OpenFluxReadLog()` — stats + log tail (free with `OpenFluxFreeString`).
Built from `mobile/ios` (`./build_ios.sh`) on package `mobile`, the same code
the Android app runs: Session, KDF context, codec fallback and openflux://
links behave the same on every client. An app outside this repository links
this core as a submodule and builds the library with the same script.
Classic profiles (one carrier, as the app has always started them):
- `OpenFluxSetEncryption(secret)`, `OpenFluxSetCodec(codec)` — for the next start.
With a secret the client runs the Session and speaks classic to a node that
does not answer it (an older or classic node); without one, classic only.
- `OpenFluxStartClient(transportType, url, socksAddr, maxToken, maxUid)` — SOCKS5 client
(a comma-separated `url` runs one carrier per document).
- `OpenFluxStartPacketTunnel(transportType, url, maxToken, maxUid)` — Network Extension
(`OpenFluxTunWritePacket` / `OpenFluxTunReadPacket`, `OpenFluxPacketTunnelConnected`,
`OpenFluxStopPacketTunnel`, `OpenFluxSetTunnelUDP`, `OpenFluxSetGeositeDirect`,
`OpenFluxDrainDirectIPs`).
Session profiles (several carriers, what a node's openflux:// link describes):
- `OpenFluxShareDecode(link)` → `{"config","context","session"}`: `session` is the
profile, ready for `OpenFluxStartSession(session, secret, socksAddr)` or
`OpenFluxStartSessionPacketTunnel(session, secret)`. Do not interpret the link in
Swift: the context and carrier names in it must reach the core unchanged.
- `OpenFluxShareEncode(configJSON)` → `{"link","config","context"}`: the core fills in
the context and drops defaults, so the link is the one every client makes.
- On failure both return `{"error","code","param"}`: `code` (`share.Code*`, e.g.
`damaged`, `unknown_transport`) is what the app words for the user, `param` the value
it is about; `error` is English detail for the log.
State and checks:
- `OpenFluxStop()`, `OpenFluxIsRunning()`, `OpenFluxIsConnected()`, `OpenFluxStatsJSON()`,
`OpenFluxMode()` (`session` / `classic`), `OpenFluxActiveTransport()`.
- `OpenFluxCaptchaPending()` (this phone's carrier), `OpenFluxRemoteCaptchaPending()` and
`OpenFluxRemoteCaptchaProxy()` (the exit's), `OpenFluxCaptchaReason()`,
`OpenFluxApplyCaptchaCookies(header)` / `OpenFluxOfferCaptchaCookies(header)`,
`OpenFluxCancelCaptcha()`, `OpenFluxSetInitialCookies(header)`, `OpenFluxSetCookieStore(path)`.
- `OpenFluxReadLog()`, `OpenFluxSetDebug(on)`, `OpenFluxSetDebugLevel(0..3)`,
`OpenFluxSetDoTResolver(spec)`; free returned strings with `OpenFluxFreeString`.
## Build + archive + export (one command)
From the repo root:
logging_flags_test.go
+10 -1
View File
@@ -1,6 +1,7 @@
package main
import (
"openflux/transport"
"reflect"
"testing"
)
@@ -51,9 +52,17 @@ func TestPickSessionContext(t *testing.T) {
{Type: "yandex", Priority: 50, URL: "https://docs/doc"},
}, "https://docs/doc"},
{"cupsonline alone", "", "http://#", []transportSpec{{Type: "cupsonline", Priority: 100, URL: "WyIxYzE0NGQwZS1lMDQw"}}, "http://#"},
// A classic cupsonline client is given the rooms as --url; the exit
// that created them ran without one. Both must land on the same
// context (they used to differ, and nothing got through).
{"classic cupsonline client with rooms as --url", "", "WyIxYzE0NGQwZS1lMDQw",
[]transportSpec{{Type: "cupsonline", Priority: 100, URL: "WyIxYzE0NGQwZS1lMDQw"}}, "http://#"},
{"direct host:port is not a context", "", "http://#", []transportSpec{
{Type: "direct", Priority: 100, URL: "203.0.113.7:9443"},
}, "http://#"},
}
for _, c := range cases {
if got := pickSessionContext(c.explicit, c.url, c.specs); got != c.want {
if got, _ := transport.KDFContexts(c.explicit, c.url, contextSources(c.specs)); got != c.want {
t.Errorf("%s: got %q, want %q", c.name, got, c.want)
}
}
main.go
+93 -65
View File
@@ -138,46 +138,15 @@ func isNumber(s string) bool {
return err == nil
}
// pickSessionContext returns the KDF salt used to derive encryption keys.
// The same value must be produced on both peers, regardless of how the
// document URL was supplied (--url, --yandex-url, [Transport] URL, ...).
//
// Priority:
//
// explicit --session-context, if non-empty
// --url globalURL, if set and not the placeholder
// transports URL of the highest-priority transport that has one,
// cupsonline aside
// fallback the placeholder "http://#"
//
// A cupsonline "URL" is the room list the exit creates when it starts and
// prints for clients, so the exit cannot know it beforehand; letting it
// into the context gave the two sides different keys.
//
// This is what the OpenFlux-Android client derives for a Session profile,
// and the fallback is what older builds used whenever --url was unset, so a
// node without any document URL (direct, oneme) keeps its old key.
func pickSessionContext(explicit, globalURL string, specs []transportSpec) string {
const placeholder = "http://#"
if explicit != "" {
return explicit
}
if globalURL != "" && globalURL != placeholder {
return globalURL
}
best := -1
// contextSources describes the carriers for transport.KDFContexts, the one
// rule every peer (CLI, Android, Desktop, iOS) derives the encryption
// context with.
func contextSources(specs []transportSpec) []transport.ContextSource {
out := make([]transport.ContextSource, len(specs))
for i, s := range specs {
if s.Type == "cupsonline" || s.URL == "" || s.URL == placeholder {
continue
}
if best < 0 || s.Priority > specs[best].Priority {
best = i
}
out[i] = transport.ContextSource{Type: s.Type, URL: s.URL, Priority: s.Priority}
}
if best >= 0 {
return specs[best].URL
}
return placeholder
return out
}
// managerRefreshLoop periodically asks the exit node for a fresh cookie jar.
@@ -202,6 +171,15 @@ func main() {
if len(os.Args) == 2 && os.Args[1] == "--node-wizard" {
os.Exit(runNodeWizard(os.Stdin, os.Stdout))
}
// The core's own openflux:// parser and builder for apps and scripts,
// so a link is read and made the same way everywhere: JSON on stdout,
// before any banner.
if len(os.Args) == 3 && os.Args[1] == "--parse-link" {
os.Exit(runParseLink(os.Args[2], os.Stdin, os.Stdout))
}
if len(os.Args) == 3 && os.Args[1] == "--make-link" {
os.Exit(runMakeLink(os.Args[2], os.Stdin, os.Stdout))
}
fmt.Print("written by p1neappleXpress\n")
role := flag.String("role", roleClient, "client | exit | bench-send | bench-sink")
@@ -655,8 +633,10 @@ DEPRECATED (removed in v2)
}
specs = buildTransportSpecs(parsed, urls, extra)
} else {
// Named after the type, as --transports and the apps name
// carriers: cookie exchange with a Session peer is by name.
specs = []transportSpec{{
Name: "primary",
Name: *transportType,
Type: *transportType,
Priority: 100,
URL: globalDocUrl,
@@ -692,9 +672,9 @@ DEPRECATED (removed in v2)
log.Fatalf("Read encryption key file: %v", err)
}
secret = strings.TrimSpace(string(b))
if len(secret) < 16 {
log.Fatalf("Encryption key from %s is too short (%d chars, need at least 16)",
*encryptionKeyFile, len(secret))
if n := utils.SecretChars(secret); n < utils.MinSecretChars {
log.Fatalf("Encryption key from %s is too short (%d chars, need at least %d)",
*encryptionKeyFile, n, utils.MinSecretChars)
}
if strings.ContainsAny(secret, "\r\n\t") {
utils.Debugf("[KEY] WARNING: secret still contains whitespace after TrimSpace; lengths may differ across platforms")
@@ -707,10 +687,11 @@ DEPRECATED (removed in v2)
}
}
sessionContext = pickSessionContext(*sessionContextFlag, globalDocUrl, specs)
sessionContext, contextAlternates := transport.KDFContexts(*sessionContextFlag, globalDocUrl, contextSources(specs))
if *encryptionKeyFile != "" {
utils.Debugf("[KEY] context=%q sha256=%s (MUST match on both peers)",
sessionContext, utils.Sha256Hex([]byte(sessionContext)))
utils.Debugf("[KEY] context=%q sha256=%s (MUST match on both peers; %d alternates tried on mismatch)",
sessionContext, utils.Sha256Hex([]byte(sessionContext)), len(contextAlternates))
log.Printf("Encryption context: sha256 %s", utils.Sha256Short([]byte(sessionContext)))
}
// Decide whether we run the full Session path (encryption + negotiate)
@@ -722,13 +703,35 @@ DEPRECATED (removed in v2)
demux *transport.PortDemux
)
// [Transport] sections in a .conf describe a multi-transport session
// just like --transports; without this they were silently ignored and
// only the single --transport ran.
if *negotiate || *transportsFlag != "" || len(confTransports) > 0 {
// configuredSession: the operator asked for a Session. [Transport]
// sections in a .conf describe one just like --transports.
//
// A classic setup (--transport=X) with a key runs as a Session too,
// with classic compatibility: a client falls back to the classic
// layering while the exit does not answer the handshake and upgrades
// once it does; an exit serves classic clients and Session clients.
// Only --negotiate is strict. Without a key only classic is possible.
configuredSession := *negotiate || *transportsFlag != "" || len(confTransports) > 0
classicCompat := false
switch {
case *role != roleClient && *role != roleExit:
case !configuredSession && secret != "":
classicCompat = true
case configuredSession && !*negotiate && *role == roleClient && len(specs) == 1:
classicCompat = true
}
if configuredSession || classicCompat {
if secret == "" {
log.Fatal("--transports/--negotiate/.conf transports require --encryption-key-file")
}
switch {
case classicCompat && isExit:
log.Printf("Mode: Session, also serving classic clients (--negotiate makes it Session-only)")
case classicCompat:
log.Printf("Mode: Session, falling back to classic while the exit does not answer the handshake")
default:
log.Printf("Mode: Session")
}
caps := transport.CapabilityIPv4 | transport.CapabilityTCP | transport.CapabilityUDP
if *role == roleClient || exitMode == tunnel.ExitModeL3 {
@@ -742,10 +745,14 @@ DEPRECATED (removed in v2)
if err != nil {
log.Fatal(err)
}
if classicCompat {
sess.SetClassic(*codec)
}
sess.SetAlternateContexts(contextAlternates)
// Build the factory that SubtypeTransportStart will use for
// dynamic transports.
factory := transportFactory(config)
factory := transportFactory(config, isExit)
managerInst = manager.New(sess, factory, secret, sessionContext)
if err := registerBootstrapTransports(managerInst, specs, config, secret, sessionContext, rooms); err != nil {
@@ -810,7 +817,8 @@ DEPRECATED (removed in v2)
exchanger = nil // cookie handling lives in the Manager
} else {
// Legacy single-transport path (no negotiate, no multi).
// Classic single-transport path without a Session: no key (the
// Session needs one), or a bench role.
var inner transport.Transport
switch *transportType {
case "boards":
@@ -847,22 +855,21 @@ DEPRECATED (removed in v2)
}
}
switch *codec {
case codecBatched:
log.Printf("Codec: batched (zstd + coalescing)")
inner = transport.NewBatchedTransport(inner)
case codecLegacy:
log.Printf("Codec: legacy (per-packet LZ4, no batching)")
inner = transport.NewCompressedTransport(inner)
}
// Either framing is accepted; the preferred one is sent until the
// peer shows which it speaks (see transport.CodecTransport).
log.Printf("Codec: %s preferred, falls back to the other framing when the peer does not answer", *codec)
inner = transport.NewCodecTransport(inner, *codec, !isExit)
if *encryptionKeyFile != "" {
encrypted, err := transport.NewEncryptedTransport(inner, secret, sessionContext, isExit)
if err != nil {
log.Fatalf("Configure encrypted transport: %v", err)
}
encrypted.SetAlternateContexts(contextAlternates)
inner = encrypted
log.Printf("Transport encryption: AES-256-GCM enabled")
} else {
log.Printf("Transport encryption: OFF (no --encryption-key-file): the carrier sees the traffic, and a peer with a key cannot talk to this one")
}
trans = inner
@@ -916,20 +923,41 @@ DEPRECATED (removed in v2)
switch *role {
case roleExit:
var printLink func()
if *shareFlag {
session := *negotiate || *transportsFlag != "" || len(confTransports) > 0
// A classic exit keeps advertising classic: older clients
// read the link too, and updated ones upgrade on their own.
session := configuredSession
host := *shareHost
if host == "" {
host = publicIPv4()
}
printLink := func() {
printLink = func() {
printShare(shareConfig(specs, session, *codec, secret, sessionContext, host, rooms))
}
// Rooms created later (a start that failed and was retried)
// or anew change the link: print it again for the clients.
for _, r := range rooms {
r.OnRoomList(func(string) { printLink() })
}
// A cupsonline exit learns its room list only once it runs. Older
// classic clients derived their key from that list; accept it as
// an alternate context. Rooms created later (a start that failed
// and was retried) or anew change the link: print it again.
var sess *transport.Session
if managerInst != nil {
sess = managerInst.Session()
}
for _, r := range rooms {
r.OnRoomList(func(packed string) {
if sess != nil && packed != "" {
sess.SetAlternateContexts([]string{packed})
}
if printLink != nil {
printLink()
}
})
if list := r.RoomList(); list != "" && sess != nil {
sess.SetAlternateContexts([]string{list})
}
}
if printLink != nil {
printLink()
}
runExit(trans, exitMode)
mobile/go.mod
+4 -2
View File
@@ -2,7 +2,10 @@ module openflux-mobile
go 1.26.4
require openflux v0.0.0
require (
golang.org/x/net v0.59.0
openflux v0.0.0
)
require (
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
@@ -35,7 +38,6 @@ require (
golang.org/x/exp v0.0.0-20250711185948-6ae5c78190dc // indirect
golang.org/x/mobile v0.0.0-20260908204917-8b95e45f8d3e // indirect
golang.org/x/mod v0.41.0 // indirect
golang.org/x/net v0.59.0 // indirect
golang.org/x/sync v0.23.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/time v0.15.0 // indirect
mobile/initial_cookies.go
+43
View File
@@ -0,0 +1,43 @@
package mobile
import (
"sync"
"openflux/transport/manager"
)
// initialCookies are cookies the app got before starting (a check passed
// with the tunnel down), applied to the Yandex carriers of the next start
// before their first request.
var initialCookies struct {
mu sync.Mutex
jar map[string]string
}
// SetInitialCookies takes a Cookie header ("a=1; b=2") for the Yandex
// carriers (yandex, vyandex, boards) of the next Start*. "" clears it.
func SetInitialCookies(cookieHeader string) {
jar := parseCookieHeader(cookieHeader)
initialCookies.mu.Lock()
initialCookies.jar = jar
initialCookies.mu.Unlock()
}
func applyInitialCookies(m *manager.Manager, specs []sessionSpec) {
initialCookies.mu.Lock()
jar := initialCookies.jar
initialCookies.mu.Unlock()
if len(jar) == 0 {
return
}
for _, s := range specs {
switch s.Type {
case "yandex", "vyandex", "boards":
if err := m.AcceptCookies(s.Name, jar); err != nil {
appendLog("[ANDROID] " + s.Name + ": начальные cookies не применились: " + err.Error())
} else {
appendLog("[ANDROID] " + s.Name + ": применены cookies, полученные до старта")
}
}
}
}
mobile/ios/bridge.go
+378
View File
@@ -0,0 +1,378 @@
//go:build ios
// Command ios is the iOS app's C library (go build -buildmode=c-archive
// -tags ios, see build_ios.sh). It exposes the C API the app calls, built
// on package mobile: the same Session, context rule, codec and link
// handling as Android, so an iOS client behaves like every other one. The
// app links this core as a submodule instead of carrying a copy.
//
// Everything here is iOS glue: C strings, the log buffer the app drains,
// DNS-over-TLS against poisoned resolvers, the packet flow of the Network
// Extension. Nothing decides how the tunnel works.
package main
/*
#include <stdlib.h>
*/
import "C"
import (
"context"
"crypto/tls"
"encoding/json"
"fmt"
"io"
"log"
"net"
"os"
"path/filepath"
"runtime/debug"
"strings"
"sync"
"time"
"unsafe"
mobile "openflux-mobile"
"openflux/transport"
"openflux/utils"
)
func main() {}
// ---- log buffer the app drains ----
type ringLog struct {
mu sync.Mutex
lines []string
}
func (r *ringLog) Write(p []byte) (int, error) {
r.add(strings.TrimRight(string(p), "\n"))
return len(p), nil
}
func (r *ringLog) add(line string) {
r.mu.Lock()
defer r.mu.Unlock()
r.lines = append(r.lines, line)
if len(r.lines) > 1000 {
r.lines = r.lines[len(r.lines)-1000:]
}
}
func (r *ringLog) drain() string {
r.mu.Lock()
defer r.mu.Unlock()
if len(r.lines) == 0 {
return ""
}
out := strings.Join(r.lines, "\n")
r.lines = r.lines[:0]
return out
}
var logbuf = &ringLog{}
func init() {
// The core's operational lines reach the app through package mobile's
// log (ReadLogs); the standard log (plain log.Printf from the core)
// goes to the ring buffer. The debug logger itself writes nowhere
// else, so no line shows up twice.
utils.SetOutput(io.Discard)
log.SetOutput(logbuf)
log.SetFlags(log.Ltime)
setupCrashCapture()
// The phone's own resolver may be poisoned for the hosts the carriers
// use; resolve over DNS-over-TLS instead.
net.DefaultResolver = &net.Resolver{PreferGo: true, Dial: dialSecureDNS}
}
// crashFile keeps the fatal-crash sink open for the process lifetime; the
// previous run's crash is shown in the log on the next launch.
var crashFile *os.File
func setupCrashCapture() {
path := filepath.Join(os.TempDir(), "oflux-crash.log")
if b, err := os.ReadFile(path); err == nil && len(b) > 0 {
logbuf.add("===== PREVIOUS CRASH (Go traceback) =====")
logbuf.add(string(b))
logbuf.add("===== END PREVIOUS CRASH =====")
}
f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0o644)
if err != nil {
return
}
crashFile = f
debug.SetCrashOutput(f, debug.CrashOptions{})
}
// OpenFluxReadLog drains buffered log lines (newline-separated). Free the
// result with OpenFluxFreeString.
//
//export OpenFluxReadLog
func OpenFluxReadLog() *C.char {
parts := make([]string, 0, 2)
if s := logbuf.drain(); s != "" {
parts = append(parts, s)
}
if s := mobile.ReadLogs(); s != "" {
parts = append(parts, s)
}
return C.CString(strings.Join(parts, "\n"))
}
//export OpenFluxFreeString
func OpenFluxFreeString(s *C.char) {
C.free(unsafe.Pointer(s))
}
// OpenFluxSetDebug turns operational logs on (level 2) or off.
//
//export OpenFluxSetDebug
func OpenFluxSetDebug(on C.int) {
level := utils.LevelOff
if on != 0 {
level = utils.LevelDebug
}
OpenFluxSetDebugLevel(C.int(level))
}
// OpenFluxSetDebugLevel sets the core's log level: 0 off, 1 packets, 2
// operational logs, 3 hexdumps (the CLI's --debug=N). Takes effect now and
// on the next start.
//
//export OpenFluxSetDebugLevel
func OpenFluxSetDebugLevel(n C.int) {
mobile.SetDebugLevel(int(n))
utils.SetLevel(int(n))
}
// ---- DNS over TLS ----
type dotServer struct {
addr string
sni string
}
func defaultDoTServers() []dotServer {
return []dotServer{
{"77.88.8.8:853", "common.dot.dns.yandex.net"},
{"8.8.8.8:853", "dns.google"},
{"1.1.1.1:853", "cloudflare-dns.com"},
}
}
var (
dotMu sync.RWMutex
dotServers = defaultDoTServers()
)
func getDoTServers() []dotServer {
dotMu.RLock()
defer dotMu.RUnlock()
return append([]dotServer(nil), dotServers...)
}
// OpenFluxSetDoTResolver sets the DNS-over-TLS upstreams: ";"-separated
// "addr[:port]@sni" entries ("1.1.1.1@cloudflare-dns.com"); port defaults
// to 853, SNI to the host. "" restores the defaults. Call before starting.
//
//export OpenFluxSetDoTResolver
func OpenFluxSetDoTResolver(spec *C.char) {
s := strings.TrimSpace(C.GoString(spec))
dotMu.Lock()
defer dotMu.Unlock()
if s == "" {
dotServers = defaultDoTServers()
return
}
var servers []dotServer
for _, part := range strings.Split(s, ";") {
part = strings.TrimSpace(part)
if part == "" {
continue
}
addr, sni := part, ""
if i := strings.LastIndex(part, "@"); i >= 0 {
addr, sni = strings.TrimSpace(part[:i]), strings.TrimSpace(part[i+1:])
}
if !strings.Contains(addr, ":") {
addr += ":853"
}
if sni == "" {
if host, _, err := net.SplitHostPort(addr); err == nil {
sni = host
} else {
sni = addr
}
}
servers = append(servers, dotServer{addr: addr, sni: sni})
}
if len(servers) > 0 {
dotServers = servers
log.Printf("[DNS] DNS-over-TLS resolvers: %v", servers)
}
}
func dialSecureDNS(ctx context.Context, _, _ string) (net.Conn, error) {
var lastErr error
for _, s := range getDoTServers() {
d := tls.Dialer{
NetDialer: &net.Dialer{Timeout: 6 * time.Second},
Config: &tls.Config{ServerName: s.sni, MinVersion: tls.VersionTLS12},
}
conn, err := d.DialContext(ctx, "tcp", s.addr)
if err == nil {
return conn, nil
}
lastErr = err
utils.Debugf("[DNS] DoT %s failed: %v", s.addr, err)
}
return nil, lastErr
}
// ---- profile settings shared by every start ----
// Start return codes.
const (
startOK = 0
startAlreadyRunning = 1
startBadTransport = 2
startTransportError = 3
startAddrInUse = 4
startPanic = 5
startBadEncryption = 6
)
var settings struct {
mu sync.Mutex
secret string
codec string
}
// OpenFluxSetEncryption sets (or clears, with "") the shared secret for the
// next start. With a secret the client runs the Session, falling back to
// the classic layering for a node that does not answer it; without one only
// classic, unencrypted, is possible.
//
//export OpenFluxSetEncryption
func OpenFluxSetEncryption(secret *C.char) {
s := ""
if secret != nil {
s = strings.TrimSpace(C.GoString(secret))
}
settings.mu.Lock()
settings.secret = s
settings.mu.Unlock()
if s == "" {
log.Printf("[BRIDGE] encryption: off")
} else {
log.Printf("[BRIDGE] encryption: secret set (%d chars)", utils.SecretChars(s))
}
}
// OpenFluxSetCodec sets the classic framing preferred for the next start,
// "batched" (default) or "legacy" (an openflux:// link's codec). The other
// one is accepted too and tried when the node does not answer.
//
//export OpenFluxSetCodec
func OpenFluxSetCodec(codec *C.char) {
c := strings.TrimSpace(C.GoString(codec))
settings.mu.Lock()
settings.codec = c
settings.mu.Unlock()
}
func currentSettings() (secret, codec string) {
settings.mu.Lock()
defer settings.mu.Unlock()
codec = settings.codec
if codec == "" {
codec = transport.CodecBatched
}
return settings.secret, codec
}
// normalizeType maps the app's transport names onto the core's.
func normalizeType(t string) string {
switch t {
case "", "yandex":
return "yandex"
case "volga":
return "vyandex"
case "mail":
return "mailru"
case "max":
return "oneme"
}
return t
}
// classicSpecs turns the app's (type, value) into a Session spec list for
// comma-separated document lists (several documents of one type): one
// carrier each, named as every client names them (type, type-2, ...).
func classicSpecs(typ, value string) []map[string]interface{} {
var specs []map[string]interface{}
for i, u := range strings.Split(value, ",") {
u = strings.TrimSpace(u)
if u == "" {
continue
}
name := typ
if i > 0 {
name = fmt.Sprintf("%s-%d", typ, i+1)
}
specs = append(specs, map[string]interface{}{
"name": name, "type": typ, "url": u, "priority": 100 - i,
})
}
return specs
}
// startCode maps package mobile's error text onto a start code.
func startCode(msg string) C.int {
switch {
case msg == "":
return startOK
case strings.Contains(msg, "занят"):
return startAddrInUse
case strings.Contains(strings.ToLower(msg), "ключ"):
return startBadEncryption
case strings.Contains(msg, "неизвестный тип"):
return startBadTransport
}
return startTransportError
}
func recoverStart(rc *C.int, where string) {
if r := recover(); r != nil {
log.Printf("[BRIDGE] recovered from a panic in %s: %v", where, r)
*rc = C.int(startPanic)
}
}
func jsonString(v interface{}) *C.char {
b, err := json.Marshal(v)
if err != nil {
return C.CString(`{"error":"marshal failed"}`)
}
return C.CString(string(b))
}
// OpenFluxMode says how traffic currently goes: "session", "classic" (the
// node does not answer the Session handshake: an older or classic node),
// or "". Free with OpenFluxFreeString.
//
//export OpenFluxMode
func OpenFluxMode() *C.char {
return C.CString(mobile.ConnectionMode())
}
// OpenFluxActiveTransport names the carrier traffic goes through now
// ("direct", "vyandex", ...), following failover. Free with
// OpenFluxFreeString.
//
//export OpenFluxActiveTransport
func OpenFluxActiveTransport() *C.char {
return C.CString(mobile.CurrentTransport())
}
mobile/ios/captcha.go
+127
View File
@@ -0,0 +1,127 @@
//go:build ios
package main
/*
#include <stdlib.h>
*/
import "C"
import (
"strings"
mobile "openflux-mobile"
)
// Checks (SmartCaptcha, a login) a carrier cannot pass on its own. The
// phone's own carriers report them as OpenFluxCaptchaPending; the exit's
// (it cannot pass them, there is no browser) as
// OpenFluxRemoteCaptchaPending: that page must be passed from the exit's
// address, so with the VPN up (every page goes through the exit) or through
// OpenFluxRemoteCaptchaProxy, and its cookies go back to the exit.
// OpenFluxCaptchaPending returns the page one of this phone's carriers
// needs passed, or "". Free with OpenFluxFreeString.
//
//export OpenFluxCaptchaPending
func OpenFluxCaptchaPending() *C.char {
if mobile.PendingCaptchaProxy() != "" {
return C.CString("")
}
return C.CString(mobile.PendingCaptchaURL())
}
// OpenFluxRemoteCaptchaPending returns the page the exit needs passed, or
// "". Free with OpenFluxFreeString.
//
//export OpenFluxRemoteCaptchaPending
func OpenFluxRemoteCaptchaPending() *C.char {
if mobile.PendingCaptchaProxy() == "" {
return C.CString("")
}
return C.CString(mobile.PendingCaptchaURL())
}
// OpenFluxRemoteCaptchaProxy returns the loopback HTTP proxy (host:port)
// whose connections leave through the exit, for a web view that can take a
// proxy (iOS 17: WKWebsiteDataStore.proxyConfigurations), or "". Free with
// OpenFluxFreeString.
//
//export OpenFluxRemoteCaptchaProxy
func OpenFluxRemoteCaptchaProxy() *C.char {
return C.CString(mobile.PendingCaptchaProxy())
}
// OpenFluxCaptchaReason is "smartcaptcha" or "login" while a check is
// pending. Free with OpenFluxFreeString.
//
//export OpenFluxCaptchaReason
func OpenFluxCaptchaReason() *C.char {
return C.CString(mobile.PendingCaptchaReason())
}
// OpenFluxApplyCaptchaCookies takes the cookies of a passed check (a
// Cookie header, "a=1; b=2") and hands them to the carrier that asked:
// this phone's, or the exit's (then they are sent to it). Returns the
// number of cookies taken, 0 on failure.
//
//export OpenFluxApplyCaptchaCookies
func OpenFluxApplyCaptchaCookies(cookies *C.char) C.int {
if cookies == nil {
return 0
}
header := C.GoString(cookies)
if mobile.SubmitCaptchaCookies(header) != "" {
return 0
}
return C.int(countCookies(header))
}
// OpenFluxOfferCaptchaCookies is OpenFluxApplyCaptchaCookies for the
// exit's check (the same call: the pending check knows whose it is).
//
//export OpenFluxOfferCaptchaCookies
func OpenFluxOfferCaptchaCookies(cookies *C.char) C.int {
return OpenFluxApplyCaptchaCookies(cookies)
}
// OpenFluxCancelCaptcha drops the pending check (the user gave up); an
// exit's check stays quiet for a while.
//
//export OpenFluxCancelCaptcha
func OpenFluxCancelCaptcha() {
mobile.CancelCaptcha()
}
// OpenFluxSetInitialCookies passes cookies got before starting (a check
// passed with the tunnel down) to the Yandex carriers of the next start.
//
//export OpenFluxSetInitialCookies
func OpenFluxSetInitialCookies(cookies *C.char) {
s := ""
if cookies != nil {
s = C.GoString(cookies)
}
mobile.SetInitialCookies(s)
}
// OpenFluxSetCookieStore keeps passed checks' cookies in a file (the app's
// shared container) across starts. Returns 0, or 1 on error.
//
//export OpenFluxSetCookieStore
func OpenFluxSetCookieStore(path *C.char) C.int {
if mobile.SetCookieStorePath(C.GoString(path)) != "" {
return 1
}
return 0
}
func countCookies(header string) int {
n := 0
for _, part := range strings.Split(header, ";") {
if name, _, ok := strings.Cut(strings.TrimSpace(part), "="); ok && name != "" {
n++
}
}
return n
}
mobile/ios/geosite.go
+153
View File
@@ -0,0 +1,153 @@
//go:build ios
package main
// GeoSite split tunneling (from the iOS app's fork, saharev1/OpenFlux). The packet-tunnel extension already proxies
// every device DNS query over DoT, so it sees both the queried domain and the
// resolved IPs. When a domain matches the geosite "direct" set, its answer IPs
// are recorded and handed to the Swift provider, which adds them to the tunnel's
// excludedRoutes so that domain's traffic goes direct — even when the IP is
// foreign (e.g. a Russian service on a foreign CDN) and thus not in the GeoIP RU
// set. This is the only way to route by domain on iOS, where the OS routes L3
// traffic and "direct" means "excluded from the tunnel".
import (
"net"
"strings"
"sync"
"golang.org/x/net/dns/dnsmessage"
)
var (
geoMu sync.RWMutex
geoSet map[string]struct{} // domain suffixes that go direct
directSeen = map[string]struct{}{}
directPend []string // IPs not yet handed to the Swift side
)
// setGeositeDirect loads the newline-separated domain-suffix list (one domain per
// line; leading "*." / "." and comments are tolerated).
func setGeositeDirect(list string) {
set := make(map[string]struct{})
for _, ln := range strings.Split(list, "\n") {
ln = strings.TrimSpace(strings.ToLower(ln))
if ln == "" || strings.HasPrefix(ln, "#") {
continue
}
ln = strings.TrimPrefix(ln, "*.")
ln = strings.TrimPrefix(ln, ".")
ln = strings.TrimSuffix(ln, ".")
if ln != "" {
set[ln] = struct{}{}
}
}
geoMu.Lock()
geoSet = set
geoMu.Unlock()
}
// geoMatch reports whether name (or any parent domain of it) is in the direct
// set. Walks label boundaries, so it is O(labels) per query regardless of set
// size.
func geoMatch(name string) bool {
name = strings.ToLower(strings.TrimSuffix(name, "."))
geoMu.RLock()
defer geoMu.RUnlock()
if len(geoSet) == 0 {
return false
}
for {
if _, ok := geoSet[name]; ok {
return true
}
i := strings.IndexByte(name, '.')
if i < 0 {
return false
}
name = name[i+1:]
}
}
// geositeNoteAnswer parses the DNS query name and answer A-records; if the name
// is a geosite-direct domain, records its IPv4 answers as pending direct routes.
// Best-effort: any parse error is silently ignored.
func geositeNoteAnswer(query, answer []byte) {
defer func() { _ = recover() }()
var pq dnsmessage.Parser
if _, err := pq.Start(query); err != nil {
return
}
q, err := pq.Question()
if err != nil {
return
}
if !geoMatch(q.Name.String()) {
return
}
var pa dnsmessage.Parser
if _, err := pa.Start(answer); err != nil {
return
}
if err := pa.SkipAllQuestions(); err != nil {
return
}
var ips []string
for {
h, err := pa.AnswerHeader()
if err != nil {
break
}
if h.Type == dnsmessage.TypeA {
r, err := pa.AResource()
if err != nil {
break
}
ips = append(ips, net.IP(r.A[:]).String())
} else {
if err := pa.SkipAnswer(); err != nil {
break
}
}
}
if len(ips) == 0 {
return
}
geoMu.Lock()
for _, ip := range ips {
if _, ok := directSeen[ip]; !ok {
directSeen[ip] = struct{}{}
directPend = append(directPend, ip)
}
}
geoMu.Unlock()
}
// drainDirectIPsCapped returns as many pending direct IPs (newline-joined) as fit
// in max bytes and re-queues the rest for the next drain.
func drainDirectIPsCapped(max int) string {
geoMu.Lock()
defer geoMu.Unlock()
if len(directPend) == 0 || max <= 0 {
return ""
}
var b strings.Builder
i := 0
for ; i < len(directPend); i++ {
extra := len(directPend[i])
if b.Len() > 0 {
extra++ // newline
}
if b.Len()+extra > max {
break
}
if b.Len() > 0 {
b.WriteByte('\n')
}
b.WriteString(directPend[i])
}
directPend = directPend[i:]
return b.String()
}
mobile/ios/packet.go
+364
View File
@@ -0,0 +1,364 @@
//go:build ios
package main
/*
#include <stdlib.h>
*/
import "C"
import (
"context"
"crypto/tls"
"encoding/binary"
"encoding/json"
"io"
"log"
"net"
"runtime/debug"
"strings"
"sync"
"sync/atomic"
"time"
"unsafe"
mobile "openflux-mobile"
"openflux/network"
"openflux/utils"
)
// Packet-tunnel (NEPacketTunnelProvider) mode: pure L3 forwarding.
//
// The device gets tunnel address 10.10.10.2, which every exit answers to,
// so its IPv4 packets go over the tunnel as they are (package mobile's
// Send/Read, the Android VPN path); no TCP stack runs in the extension.
// DNS (UDP 53) is answered locally over DNS-over-TLS; other UDP goes over
// the tunnel when enabled, otherwise is refused with ICMP so apps fall back
// from QUIC to TCP at once.
var pt struct {
mu sync.Mutex
on bool
outQ chan []byte
ctx context.Context
cancel context.CancelFunc
}
// tunnelUDP: forward non-DNS UDP over the tunnel (the Session negotiates
// whether the exit takes UDP; a classic exit may not).
var tunnelUDP atomic.Bool
//export OpenFluxSetTunnelUDP
func OpenFluxSetTunnelUDP(on C.int) { tunnelUDP.Store(on != 0) }
// extensionLimits keeps the Network Extension under its 50 MB cap: a Go
// heap limit with headroom for what lives outside it (TLS, WebSocket
// buffers, the runtime), and the phone resource profile of the carriers.
func extensionLimits() {
debug.SetMemoryLimit(32 << 20)
debug.SetGCPercent(100)
mobile.SetLowMemory(true)
}
// OpenFluxStartPacketTunnel starts the packet tunnel for a classic profile
// (see OpenFluxStartClient for the arguments; the secret comes from
// OpenFluxSetEncryption). Returns a start code.
//
//export OpenFluxStartPacketTunnel
func OpenFluxStartPacketTunnel(transportType, url, maxToken, maxUid *C.char) (rc C.int) {
defer recoverStart(&rc, "OpenFluxStartPacketTunnel")
typ := normalizeType(C.GoString(transportType))
value := strings.TrimSpace(C.GoString(url))
secret, codec := currentSettings()
if typ == "direct" && secret == "" {
log.Printf("[PKT] direct needs an encryption key")
return startBadEncryption
}
return startPacketTunnel(func() string {
if specs := classicSpecs(typ, value); len(specs) > 1 {
if secret == "" {
return "для нескольких документов нужен ключ шифрования"
}
b, _ := json.Marshal(specs)
return mobile.StartSession(string(b), secret)
}
return mobile.Start(typ, value, secret, codec, C.GoString(maxToken), C.GoString(maxUid))
})
}
// OpenFluxStartSessionPacketTunnel starts the packet tunnel for a Session
// profile (specsJSON as for OpenFluxStartSession).
//
//export OpenFluxStartSessionPacketTunnel
func OpenFluxStartSessionPacketTunnel(specsJSON, secret *C.char) (rc C.int) {
defer recoverStart(&rc, "OpenFluxStartSessionPacketTunnel")
specs := C.GoString(specsJSON)
key := strings.TrimSpace(C.GoString(secret))
return startPacketTunnel(func() string { return mobile.StartSession(specs, key) })
}
func startPacketTunnel(start func() string) C.int {
pt.mu.Lock()
defer pt.mu.Unlock()
if pt.on {
return startAlreadyRunning
}
extensionLimits()
if msg := start(); msg != "" {
log.Printf("[PKT] start: %s", msg)
return startCode(msg)
}
// 2048 packets (about 3 MB at full size) absorb a download burst
// without drops, within the extension's budget.
pt.outQ = make(chan []byte, 2048)
pt.ctx, pt.cancel = context.WithCancel(context.Background())
pt.on = true
go pumpFromTunnel(pt.ctx, pt.outQ)
log.Printf("[PKT] packet tunnel started (mode %q)", mobile.ConnectionMode())
return startOK
}
// pumpFromTunnel moves packets the exit sent into the device queue.
func pumpFromTunnel(ctx context.Context, outQ chan []byte) {
for ctx.Err() == nil {
p := mobile.ReadTimeout(250)
if p == nil {
continue
}
select {
case outQ <- p:
default: // device side not keeping up: drop, TCP retransmits
}
}
}
// OpenFluxTunWritePacket takes one packet from the device.
//
//export OpenFluxTunWritePacket
func OpenFluxTunWritePacket(buf *C.char, length C.int) {
defer func() { _ = recover() }()
if buf == nil || length < 20 {
return
}
pt.mu.Lock()
on, outQ := pt.on, pt.outQ
pt.mu.Unlock()
if !on {
return
}
pkt := C.GoBytes(unsafe.Pointer(buf), length)
if pkt[0]>>4 != 4 {
return
}
switch pkt[9] {
case 6: // TCP
send(pkt)
case 17: // UDP
ihl := int(pkt[0]&0x0f) * 4
if len(pkt) < ihl+8 {
return
}
switch {
case binary.BigEndian.Uint16(pkt[ihl+2:ihl+4]) == 53:
select {
case dnsSem <- struct{}{}:
go func() { defer func() { <-dnsSem }(); handleDNSPacket(pkt, outQ) }()
default: // too many in flight: the client retries
}
case tunnelUDP.Load():
send(pkt)
default:
sendICMPPortUnreachable(pkt, outQ)
}
}
}
func send(pkt []byte) {
if msg := mobile.Send(pkt); msg != "" && utils.Throttled("pkt.send", 10*time.Second) {
utils.Debugf("[PKT] send: %s", msg)
}
}
// dnsSem caps concurrent DNS-over-TLS resolutions.
var dnsSem = make(chan struct{}, 16)
// OpenFluxTunReadPacket blocks for the next packet to the device and
// returns its length (0 once the tunnel stops).
//
//export OpenFluxTunReadPacket
func OpenFluxTunReadPacket(buf *C.char, max C.int) C.int {
pt.mu.Lock()
outQ, ctx := pt.outQ, pt.ctx
pt.mu.Unlock()
if outQ == nil || ctx == nil {
return 0
}
select {
case data := <-outQ:
n := min(len(data), int(max))
dst := unsafe.Slice((*byte)(unsafe.Pointer(buf)), int(max))
copy(dst[:n], data[:n])
return C.int(n)
case <-ctx.Done():
return 0
}
}
// OpenFluxPacketTunnelConnected is 1 while the tunnel reaches the exit;
// the provider polls it to drive reasserting across carrier reconnects.
//
//export OpenFluxPacketTunnelConnected
func OpenFluxPacketTunnelConnected() C.int {
pt.mu.Lock()
on := pt.on
pt.mu.Unlock()
if on && mobile.IsConnected() {
return 1
}
return 0
}
//export OpenFluxStopPacketTunnel
func OpenFluxStopPacketTunnel() {
pt.mu.Lock()
defer pt.mu.Unlock()
if !pt.on {
return
}
pt.cancel()
mobile.Stop()
pt.outQ, pt.ctx, pt.cancel = nil, nil, nil
pt.on = false
log.Printf("[PKT] packet tunnel stopped")
}
// OpenFluxSetGeositeDirect loads the newline-separated domain suffixes whose
// traffic goes direct (GeoSite split tunneling).
//
//export OpenFluxSetGeositeDirect
func OpenFluxSetGeositeDirect(list *C.char) {
setGeositeDirect(C.GoString(list))
}
// OpenFluxDrainDirectIPs copies pending direct IPs (newline-joined) into
// buf up to max bytes and returns the byte count; the provider adds them to
// excludedRoutes. What does not fit stays queued.
//
//export OpenFluxDrainDirectIPs
func OpenFluxDrainDirectIPs(buf *C.char, max C.int) C.int {
s := drainDirectIPsCapped(int(max))
if s == "" {
return 0
}
dst := unsafe.Slice((*byte)(unsafe.Pointer(buf)), int(max))
return C.int(copy(dst, s))
}
// sendICMPPortUnreachable answers a UDP datagram that is not forwarded, so
// the sender falls back to TCP at once instead of timing out.
func sendICMPPortUnreachable(orig []byte, outQ chan []byte) {
ihl := int(orig[0]&0x0f) * 4
if len(orig) < ihl+8 {
return
}
quote := orig[:ihl+8]
icmp := make([]byte, 8+len(quote))
icmp[0], icmp[1] = 3, 3 // destination unreachable, port unreachable
copy(icmp[8:], quote)
ck := network.IPChecksum(icmp)
icmp[2], icmp[3] = byte(ck>>8), byte(ck)
total := 20 + len(icmp)
ip := make([]byte, total)
ip[0] = 0x45
binary.BigEndian.PutUint16(ip[2:4], uint16(total))
ip[8], ip[9] = 64, 1
copy(ip[12:16], orig[16:20])
copy(ip[16:20], orig[12:16])
ck2 := network.IPChecksum(ip[:20])
ip[10], ip[11] = byte(ck2>>8), byte(ck2)
copy(ip[20:], icmp)
select {
case outQ <- ip:
default:
}
}
// handleDNSPacket answers a device DNS query over DNS-over-TLS.
func handleDNSPacket(req []byte, outQ chan []byte) {
defer func() { _ = recover() }()
ihl := int(req[0]&0x0f) * 4
if len(req) < ihl+8 {
return
}
query := req[ihl+8:]
if len(query) == 0 {
return
}
answer, err := dnsOverTLS(query)
if err != nil || len(answer) == 0 {
if utils.Throttled("pkt.dns", 30*time.Second) {
log.Printf("[DNS] DNS-over-TLS failed: %v", err)
}
return
}
geositeNoteAnswer(query, answer)
udpLen := 8 + len(answer)
total := ihl + udpLen
resp := make([]byte, total)
resp[0], resp[1] = req[0], req[1]
binary.BigEndian.PutUint16(resp[2:4], uint16(total))
resp[8], resp[9] = 64, 17
copy(resp[12:16], req[16:20])
copy(resp[16:20], req[12:16])
ipck := network.IPChecksum(resp[:20])
resp[10], resp[11] = byte(ipck>>8), byte(ipck)
copy(resp[ihl:ihl+2], req[ihl+2:ihl+4])
copy(resp[ihl+2:ihl+4], req[ihl:ihl+2])
binary.BigEndian.PutUint16(resp[ihl+4:ihl+6], uint16(udpLen))
copy(resp[ihl+8:], answer)
select {
case outQ <- resp:
default:
}
}
func dnsOverTLS(query []byte) ([]byte, error) {
var lastErr error
for _, s := range getDoTServers() {
ans, err := dotQueryOne(s, query)
if err == nil {
return ans, nil
}
lastErr = err
}
return nil, lastErr
}
func dotQueryOne(s dotServer, query []byte) ([]byte, error) {
d := tls.Dialer{
NetDialer: &net.Dialer{Timeout: 6 * time.Second},
Config: &tls.Config{ServerName: s.sni, MinVersion: tls.VersionTLS12},
}
conn, err := d.DialContext(context.Background(), "tcp", s.addr)
if err != nil {
return nil, err
}
defer conn.Close()
_ = conn.SetDeadline(time.Now().Add(6 * time.Second))
var lp [2]byte
binary.BigEndian.PutUint16(lp[:], uint16(len(query)))
if _, err := conn.Write(append(lp[:], query...)); err != nil {
return nil, err
}
hdr := make([]byte, 2)
if _, err := io.ReadFull(conn, hdr); err != nil {
return nil, err
}
ans := make([]byte, binary.BigEndian.Uint16(hdr))
if _, err := io.ReadFull(conn, ans); err != nil {
return nil, err
}
return ans, nil
}
mobile/ios/proxy.go
+120
View File
@@ -0,0 +1,120 @@
//go:build ios
package main
/*
#include <stdlib.h>
*/
import "C"
import (
"encoding/json"
"fmt"
"log"
"strings"
"time"
mobile "openflux-mobile"
)
// The in-app core: a local SOCKS5 proxy over the tunnel (the app's own
// connection test and captcha pages), package mobile's Proxy mode.
var proxyStarted time.Time
// OpenFluxStartClient starts the SOCKS5 client on socksAddr for a classic
// profile: transportType ("yandex", "vyandex"/"volga", "boards", "mailru",
// "cupsonline", "direct", "oneme") and url (the document, the room list,
// host:port for direct; a comma-separated list runs one carrier per
// document). With a secret (OpenFluxSetEncryption) it is a Session that
// speaks classic to a node that does not answer the handshake. Returns a
// start code.
//
//export OpenFluxStartClient
func OpenFluxStartClient(transportType, url, socksAddr, maxToken, maxUid *C.char) (rc C.int) {
defer recoverStart(&rc, "OpenFluxStartClient")
if mobile.ProxyIsRunning() {
return startAlreadyRunning
}
typ := normalizeType(C.GoString(transportType))
value := strings.TrimSpace(C.GoString(url))
addr := C.GoString(socksAddr)
secret, codec := currentSettings()
if typ == "direct" && secret == "" {
log.Printf("[BRIDGE] direct needs an encryption key")
return startBadEncryption
}
var msg string
if specs := classicSpecs(typ, value); len(specs) > 1 {
if secret == "" {
log.Printf("[BRIDGE] several documents need an encryption key (a Session)")
return startBadEncryption
}
b, _ := json.Marshal(specs)
msg = mobile.StartSessionProxy(string(b), secret, addr, "", "", "")
} else {
msg = mobile.StartProxy(typ, value, secret, codec, C.GoString(maxToken), C.GoString(maxUid), addr, "", "", "")
}
if msg != "" {
log.Printf("[BRIDGE] start: %s", msg)
return startCode(msg)
}
proxyStarted = time.Now()
return startOK
}
// OpenFluxStartSession starts the SOCKS5 client for a Session profile:
// specsJSON is what OpenFluxShareDecode returns as "session" (or package
// mobile's StartSession takes).
//
//export OpenFluxStartSession
func OpenFluxStartSession(specsJSON, secret, socksAddr *C.char) (rc C.int) {
defer recoverStart(&rc, "OpenFluxStartSession")
if mobile.ProxyIsRunning() {
return startAlreadyRunning
}
msg := mobile.StartSessionProxy(C.GoString(specsJSON), strings.TrimSpace(C.GoString(secret)), C.GoString(socksAddr), "", "", "")
if msg != "" {
log.Printf("[BRIDGE] start: %s", msg)
return startCode(msg)
}
proxyStarted = time.Now()
return startOK
}
//export OpenFluxStop
func OpenFluxStop() {
mobile.StopProxy()
}
//export OpenFluxIsRunning
func OpenFluxIsRunning() C.int {
if mobile.ProxyIsRunning() {
return 1
}
return 0
}
//export OpenFluxIsConnected
func OpenFluxIsConnected() C.int {
if mobile.ProxyIsConnected() {
return 1
}
return 0
}
// OpenFluxStatsJSON returns the SOCKS client's state as JSON. Free with
// OpenFluxFreeString.
//
//export OpenFluxStatsJSON
func OpenFluxStatsJSON() *C.char {
running := mobile.ProxyIsRunning()
uptime := int64(0)
if running {
uptime = int64(time.Since(proxyStarted) / time.Second)
}
return C.CString(fmt.Sprintf(
`{"running":%t,"connected":%t,"bytesSent":%d,"bytesReceived":%d,"uptimeSec":%d,"mode":%q,"transport":%q}`,
running, mobile.ProxyIsConnected(), mobile.ProxyBytesSent(), mobile.ProxyBytesReceived(), uptime,
mobile.ConnectionMode(), mobile.CurrentTransport()))
}
mobile/ios/share.go
+55
View File
@@ -0,0 +1,55 @@
//go:build ios
package main
/*
#include <stdlib.h>
*/
import "C"
import (
mobile "openflux-mobile"
"openflux/share"
)
// Links are read and made by the core only: the app hands the string (or
// the configuration) over and gets share.Result back as JSON, the same
// answer Desktop, Android and the CLI get. The app words its code.
type shareResult struct {
share.Result
// Session is the profile ready for OpenFluxStartSession /
// OpenFluxStartSessionPacketTunnel (with the link's secret): every
// carrier, its name, priority and address, and the context. A
// one-carrier link works there too, speaking classic to a classic node.
Session string `json:"session,omitempty"`
}
// OpenFluxShareDecode reads an openflux:// link: {"config":...,
// "context":...,"session":...} or {"error":...,"code":...,"param":...}.
// Free with OpenFluxFreeString.
//
//export OpenFluxShareDecode
func OpenFluxShareDecode(link *C.char) *C.char {
if link == nil {
return C.CString(share.Read("").JSON())
}
s := C.GoString(link)
r := shareResult{Result: share.Read(s)}
if r.Config != nil {
r.Session, _ = mobile.ShareSessionSpecs(s)
}
return jsonString(r)
}
// OpenFluxShareEncode builds the link for a share.Config JSON the way
// every client exports one: {"link":...,"config":...,"context":...} or the
// error. The core validates it, so no invalid link leaves the app.
//
//export OpenFluxShareEncode
func OpenFluxShareEncode(cfgJSON *C.char) *C.char {
if cfgJSON == nil {
return C.CString(share.MakeJSON("").JSON())
}
return C.CString(share.MakeJSON(C.GoString(cfgJSON)).JSON())
}
mobile/mobile.go
+92 -29
View File
@@ -4,11 +4,13 @@
package mobile
import (
"encoding/json"
"fmt"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
"openflux/transport"
"openflux/transport/cupsonline"
@@ -44,6 +46,20 @@ type packetClient struct {
transport transport.Transport
packets [][]byte
logs []string
// arrived is signalled when a packet is queued, for ReadTimeout.
arrived chan struct{}
}
// lowMemory selects the phone resource profile for the carriers of the
// next start (see SetLowMemory).
var lowMemory atomic.Bool
// SetLowMemory picks the resource profile for the next Start*: on, the
// carriers use small queues and buffers (Volga's slim profile) - what an
// iOS Network Extension (50 MB for the whole process) needs. The wire
// format is the same either way.
func SetLowMemory(on bool) {
lowMemory.Store(on)
}
func appendLog(message string) {
@@ -89,6 +105,9 @@ func startPacket(build func() (transport.Transport, error)) string {
client.running = true
client.packets = nil
client.logs = nil
if client.arrived == nil {
client.arrived = make(chan struct{}, 1)
}
client.mu.Unlock()
utils.SetLevel(int(debugLevel.Load()))
@@ -119,7 +138,12 @@ func startPacket(build func() (transport.Transport, error)) string {
client.packets = client.packets[1:]
}
client.packets = append(client.packets, packet)
arrived := client.arrived
client.mu.Unlock()
select {
case arrived <- struct{}{}:
default:
}
})
if err := trans.Start(); err != nil {
return fail(err)
@@ -135,23 +159,41 @@ func validateClassic(transportType, documentURL, encryptionSecret string) string
if transportType != "oneme" && documentURL == "" {
return "Ссылка на документ не указана"
}
if encryptionSecret != "" && len(encryptionSecret) < 16 {
return "Ключ шифрования должен содержать не менее 16 символов"
if encryptionSecret != "" && utils.SecretChars(encryptionSecret) < utils.MinSecretChars {
return fmt.Sprintf("Ключ шифрования должен содержать не менее %d символов", utils.MinSecretChars)
}
return ""
}
// classicTransport builds the single-transport stack: carrier, codec and
// optional encryption, the same layering as the CLI without --negotiate.
// exit selects the exit node's side (the phone as the exit).
// classicTransport builds a classic single-transport profile (the CLI's
// --transport=X). exit selects the exit node's side (the phone as the exit).
//
// With a key it is a Session with the classic layering next to it, as the
// CLI does: the client speaks classic until the exit answers the Session
// handshake, then switches (so a classic profile works against every node,
// old and new); a classic exit serves both kinds of client. Without a key
// only classic is possible. Either way the codec is the preferred framing,
// not a requirement: both are accepted and the other one is tried when the
// peer stays silent.
func classicTransport(transportType, documentURL, encryptionSecret, codec, maxToken, maxUid string, exit bool) (transport.Transport, error) {
if transportType == "" {
transportType = "yandex"
}
appendLog(fmt.Sprintf("[ANDROID] Запуск транспорта %s", transportType))
params := classicParams(transportType, documentURL, maxToken, maxUid)
if encryptionSecret != "" {
specs, err := json.Marshal([]sessionSpec{{
Name: transportType, Type: transportType, URL: documentURL, Priority: 100, Params: params,
}})
if err != nil {
return nil, err
}
t, _, err := buildSessionWith(string(specs), encryptionSecret, exit, sessionOptions{classic: true, codec: codec})
return t, err
}
config := transport.DefaultConfig()
inner, err := newRawTransport(transportType, documentURL,
map[string]interface{}{"token": maxToken, "uid": maxUid}, config, exit)
inner, err := newRawTransport(transportType, documentURL, params, config, exit)
if err != nil {
return nil, err
}
@@ -160,30 +202,23 @@ func classicTransport(transportType, documentURL, encryptionSecret, codec, maxTo
if exit {
addExitRoom(transportType, inner)
}
// The codec sits under the (absent) encryption layer; see
// transport.CodecTransport for how the framing is agreed on.
inner = transport.NewCodecTransport(inner, codec, !exit)
appendLog("[ANDROID] Шифрование транспорта отключено (ключ не задан): узел с ключом с этим профилем не заговорит")
return inner, nil
}
// App-layer codec, same as the CLI's --codec flag. Both peers must use
// the same one. Applied before encryption so it compresses plaintext
// rather than ciphertext.
if codec == "legacy" {
inner = transport.NewCompressedTransport(inner)
} else {
inner = transport.NewBatchedTransport(inner)
// classicParams are the carrier parameters of a classic profile: the MAX
// token and uid, or for direct the address, dialled by a client and
// listened on by an exit.
func classicParams(transportType, documentURL, maxToken, maxUid string) map[string]interface{} {
params := map[string]interface{}{"token": maxToken, "uid": maxUid}
if transportType == "direct" {
params["dial"] = documentURL
params["listen"] = documentURL
}
if encryptionSecret == "" {
appendLog("[ANDROID] Шифрование транспорта отключено (ключ не задан)")
return inner, nil
}
// Same context as the core: the document URL, or "http://#" when there
// isn't one (oneme, direct). Both peers must derive the same context or
// the encrypted channel just won't work.
encrypted, err := transport.NewEncryptedTransport(inner, encryptionSecret,
classicContext(documentURL), exit)
if err != nil {
return nil, err
}
appendLog("[ANDROID] Шифрование транспорта: AES-256-GCM включено")
return encrypted, nil
return params
}
// newRawTransport builds one carrier, like the CLI's transportFactory.
@@ -195,10 +230,16 @@ func newRawTransport(typ, url string, params map[string]interface{}, config tran
v, _ := params[key].(string)
return v
}
if lowMemory.Load() {
config.MaxQueueSize = 512
}
switch typ {
case "", "yandex":
return yandex.NewYandexDocsTransport(url, config), nil
case "vyandex":
if lowMemory.Load() {
return yandex.NewYandexVolgaTransportWithConfig(url, config, yandex.SlimVolgaConfig()), nil
}
return yandex.NewYandexVolgaTransport(url, config), nil
case "boards":
return yandex.NewBoardsTransport(url, config), nil
@@ -280,6 +321,28 @@ func Read() []byte {
return packet
}
// ReadTimeout is Read that waits up to timeoutMs for a packet, for callers
// that block on the tunnel (the iOS packet flow) instead of polling.
func ReadTimeout(timeoutMs int) []byte {
if p := Read(); p != nil {
return p
}
client.mu.Lock()
arrived := client.arrived
client.mu.Unlock()
if arrived == nil {
return nil
}
timer := time.NewTimer(time.Duration(timeoutMs) * time.Millisecond)
defer timer.Stop()
select {
case <-arrived:
return Read()
case <-timer.C:
return nil
}
}
// ReadLogs returns and clears the pending log lines.
func ReadLogs() string {
client.mu.Lock()
mobile/node.go
+2 -1
View File
@@ -282,7 +282,8 @@ func NodeVerify(specsJSON, secret, expectHost string, timeoutSec int) string {
}
appendLog("[NODE] Проверка канала: подключение")
trans, sess, err := buildSessionWith(specsJSON, secret, false)
// Strict: the check is that the new node answers the Session.
trans, sess, err := buildSessionWith(specsJSON, secret, false, sessionOptions{strict: true})
if err != nil {
return failure(err, nil)
}
mobile/route.go
+14
View File
@@ -47,3 +47,17 @@ func CurrentTransport() string {
}
return ""
}
// ConnectionMode says how traffic currently goes: "session" (the
// authenticated Session), "classic" (the exit does not answer the Session
// handshake: an older or classic node, or it has not answered yet), or ""
// (not connected / no key).
func ConnectionMode() string {
route.mu.Lock()
s := route.session
route.mu.Unlock()
if s == nil {
return ""
}
return s.Mode()
}
mobile/session.go
+76 -40
View File
@@ -6,11 +6,13 @@ import (
"openflux/transport"
"openflux/transport/manager"
"openflux/utils"
)
// sessionSpec is one transport of a Session profile, as the app sends it.
// Names must match the exit's (the CLI names --transports entries after
// their type) because cookie exchange is addressed by name.
// their type) because cookie exchange is addressed by name; the Manager
// also matches carriers by document URL and type when they differ.
type sessionSpec struct {
Name string `json:"name"`
Type string `json:"type"`
@@ -19,30 +21,62 @@ type sessionSpec struct {
Params map[string]interface{} `json:"params"`
}
// sessionOptions tunes buildSessionWith beyond the specs.
type sessionOptions struct {
// classic enables the classic layering next to the Session (see
// transport.Session.SetClassic): a classic profile's client falls back
// to it, a classic exit serves classic clients. codec is its
// preferred framing.
classic bool
codec string
// strict turns the classic fallback a single-carrier Session client
// gets by default off.
strict bool
}
// parseSessionSpecs reads what the app sends: a JSON array of sessionSpec,
// or {"context": ..., "transports": [...]} when the profile carries the
// exit's encryption context explicitly (imported from an openflux:// link).
// Without one the context is derived with sessionContext.
func parseSessionSpecs(specsJSON string) ([]sessionSpec, string, error) {
// The context is picked by transport.KDFContexts, the rule every peer uses;
// alternates are what other builds may have derived instead.
func parseSessionSpecs(specsJSON string) (specs []sessionSpec, context string, alternates []string, err error) {
var wrapped struct {
Context string `json:"context"`
Transports []sessionSpec `json:"transports"`
}
var specs []sessionSpec
if err := json.Unmarshal([]byte(specsJSON), &specs); err != nil {
if err := json.Unmarshal([]byte(specsJSON), &wrapped); err != nil {
return nil, "", fmt.Errorf("список транспортов: %w", err)
return nil, "", nil, fmt.Errorf("список транспортов: %w", err)
}
specs = wrapped.Transports
}
if len(specs) == 0 {
return nil, "", fmt.Errorf("список транспортов пуст")
return nil, "", nil, fmt.Errorf("список транспортов пуст")
}
context := wrapped.Context
if context == "" {
context = sessionContext(specs)
seen := make(map[string]int)
for i := range specs {
if specs[i].Name == "" {
seen[specs[i].Type]++
specs[i].Name = specs[i].Type
if n := seen[specs[i].Type]; n > 1 {
specs[i].Name = fmt.Sprintf("%s-%d", specs[i].Type, n)
}
}
}
return specs, context, nil
context, alternates = transport.KDFContexts(wrapped.Context, "", contextSources(specs))
return specs, context, alternates, nil
}
func contextSources(specs []sessionSpec) []transport.ContextSource {
out := make([]transport.ContextSource, len(specs))
for i, s := range specs {
url := s.URL
if s.Type == "direct" {
url = ""
}
out[i] = transport.ContextSource{Type: s.Type, URL: url, Priority: s.Priority}
}
return out
}
// buildSession mirrors the CLI client's --negotiate / --transports path, so
@@ -50,19 +84,19 @@ func parseSessionSpecs(specsJSON string) ([]sessionSpec, string, error) {
// specsJSON is what parseSessionSpecs reads. exit builds the exit node's
// side (the phone as an l4 exit).
func buildSession(specsJSON, secret string, exit bool) (transport.Transport, error) {
t, _, err := buildSessionWith(specsJSON, secret, exit)
t, _, err := buildSessionWith(specsJSON, secret, exit, sessionOptions{})
return t, err
}
// buildSessionWith is buildSession that also returns the Session, for
// callers that need per-transport state.
func buildSessionWith(specsJSON, secret string, exit bool) (transport.Transport, *transport.Session, error) {
specs, context, err := parseSessionSpecs(specsJSON)
func buildSessionWith(specsJSON, secret string, exit bool, opt sessionOptions) (transport.Transport, *transport.Session, error) {
specs, context, alternates, err := parseSessionSpecs(specsJSON)
if err != nil {
return nil, nil, err
}
if len(secret) < 16 {
return nil, nil, fmt.Errorf("для режима Session нужен ключ шифрования не короче 16 символов")
if utils.SecretChars(secret) < utils.MinSecretChars {
return nil, nil, fmt.Errorf("для режима Session нужен ключ шифрования не короче %d символов", utils.MinSecretChars)
}
// Like the CLI: an l4 exit terminates flows in gVisor and has no raw
@@ -78,6 +112,20 @@ func buildSessionWith(specsJSON, secret string, exit bool) (transport.Transport,
if err != nil {
return nil, nil, err
}
// A single-carrier client falls back to the classic layering while the
// exit does not answer the handshake (an exit that predates Session or
// runs classic) and upgrades once it does. A classic exit also serves
// classic clients.
switch {
case opt.classic:
sess.SetClassic(opt.codec)
case !exit && !opt.strict && len(specs) == 1:
sess.SetClassic(transport.CodecBatched)
}
sess.SetAlternateContexts(alternates)
appendLog(fmt.Sprintf("[ANDROID] Session: контекст шифрования sha256 %s (запасных: %d)",
utils.Sha256Short([]byte(context)), len(alternates)))
m := manager.New(sess, nil, secret, context)
config := transport.DefaultConfig()
keys := make(map[string]string)
@@ -90,6 +138,15 @@ func buildSessionWith(specsJSON, secret string, exit bool) (transport.Transport,
}
if exit {
addExitRoom(spec.Name, raw)
// A cupsonline exit learns its rooms at start; older classic
// clients derived their key from that list.
if r, ok := raw.(interface{ OnRoomList(func(string)) }); ok {
r.OnRoomList(func(packed string) {
if packed != "" {
sess.SetAlternateContexts([]string{packed})
}
})
}
}
if err := sess.AddTransport(spec.Name, raw, secret, context, spec.Priority); err != nil {
return nil, nil, err
@@ -98,6 +155,9 @@ func buildSessionWith(specsJSON, secret string, exit bool) (transport.Transport,
if err := m.Add(spec.Name, spec.Type, raw, spec.Priority, provider); err != nil {
return nil, nil, err
}
if spec.Type != "direct" && spec.Type != "oneme" {
m.SetURL(spec.Name, spec.URL)
}
if provider != nil {
keys[spec.Name] = spec.Type + " " + spec.URL
}
@@ -105,6 +165,7 @@ func buildSessionWith(specsJSON, secret string, exit bool) (transport.Transport,
}
sess.SetControlHandler(m.DispatchControl)
setSessionRoute(sess, types)
applyInitialCookies(m, specs)
if exit {
// The exit relays its own checks to the client itself (AuthRequired);
// the phone's UI can still pass them locally.
@@ -122,28 +183,3 @@ func buildSessionWith(specsJSON, secret string, exit bool) (transport.Transport,
appendLog("[ANDROID] Session: шифрование AES-256-GCM, согласование с нодой")
return demux, sess, nil
}
// sessionContext is the encryption context, derived as the core's
// pickSessionContext does for an exit without --url: the document URL of the
// highest-priority transport that has one, cupsonline aside (its room list
// only exists once the exit is up), else "http://#". Equal priorities keep
// the first transport, as the core does.
func sessionContext(specs []sessionSpec) string {
best := -1
for i, s := range specs {
if s.Type == "cupsonline" || s.URL == "" || s.URL == placeholderURL {
continue
}
if best < 0 || s.Priority > specs[best].Priority {
best = i
}
}
if best >= 0 {
return specs[best].URL
}
return placeholderURL
}
// placeholderURL is the core's --url default, the context of a channel that
// has no document URL.
const placeholderURL = "http://#"
mobile/session_test.go
+45 -12
View File
@@ -1,6 +1,7 @@
package mobile
import (
"encoding/json"
"fmt"
"net"
"testing"
@@ -64,16 +65,29 @@ func TestBuildSessionConnectsOverDirect(t *testing.T) {
}
}
func contextOf(t *testing.T, specs []sessionSpec) string {
t.Helper()
b, err := json.Marshal(specs)
if err != nil {
t.Fatal(err)
}
_, ctx, _, err := parseSessionSpecs(string(b))
if err != nil {
t.Fatal(err)
}
return ctx
}
func TestSessionContextPrefersHighestPriorityURL(t *testing.T) {
specs := []sessionSpec{
{Name: "mailru", URL: "https://cloud.example/m", Priority: 10},
{Name: "direct", Priority: 100},
{Name: "yandex", URL: "https://docs.example/d", Priority: 50},
{Name: "mailru", Type: "mailru", URL: "https://cloud.example/m", Priority: 10},
{Name: "direct", Type: "direct", Priority: 100},
{Name: "yandex", Type: "yandex", URL: "https://docs.example/d", Priority: 50},
}
if got := sessionContext(specs); got != "https://docs.example/d" {
if got := contextOf(t, specs); got != "https://docs.example/d" {
t.Fatalf("context = %q", got)
}
if got := sessionContext([]sessionSpec{{Name: "direct"}}); got != "http://#" {
if got := contextOf(t, []sessionSpec{{Name: "direct", Type: "direct"}}); got != "http://#" {
t.Fatalf("context without URLs = %q", got)
}
}
@@ -85,19 +99,38 @@ func TestSessionContextSkipsCupsonline(t *testing.T) {
{Name: "cupsonline", Type: "cupsonline", URL: "room-list", Priority: 100},
{Name: "yandex", Type: "yandex", URL: "https://docs.example/d", Priority: 50},
}
if got := sessionContext(specs); got != "https://docs.example/d" {
if got := contextOf(t, specs); got != "https://docs.example/d" {
t.Fatalf("context = %q", got)
}
if got := sessionContext(specs[:1]); got != "http://#" {
if got := contextOf(t, specs[:1]); got != "http://#" {
t.Fatalf("context of a cupsonline-only session = %q", got)
}
}
func TestClassicContextFallsBackToPlaceholder(t *testing.T) {
if got := classicContext(""); got != "http://#" {
t.Fatalf("context without a document = %q", got)
// A link's explicit context wins, and the derived one stays an alternate.
func TestSessionContextFromLink(t *testing.T) {
_, ctx, alts, err := parseSessionSpecs(`{"context":"https://x/ctx","transports":[{"name":"yandex","type":"yandex","url":"https://docs.example/d","priority":100}]}`)
if err != nil {
t.Fatal(err)
}
if got := classicContext("https://docs.example/d"); got != "https://docs.example/d" {
t.Fatalf("context = %q", got)
if ctx != "https://x/ctx" {
t.Fatalf("context = %q", ctx)
}
found := false
for _, a := range alts {
found = found || a == "https://docs.example/d"
}
if !found {
t.Fatalf("derived context missing from alternates %q", alts)
}
}
// A classic exit's link names the context the classic client derives.
func TestClassicShareContext(t *testing.T) {
if c := exitShareClassic("mailru", "https://cloud.example/m", "0123456789abcdef", "batched"); c.Context != "https://cloud.example/m" {
t.Fatalf("mailru context = %q", c.Context)
}
if c := exitShareClassic("cupsonline", "room-list", "0123456789abcdef", "batched"); c.Context != "http://#" {
t.Fatalf("cupsonline context = %q", c.Context)
}
}
mobile/share.go
+69 -15
View File
@@ -3,10 +3,12 @@ package mobile
import (
"encoding/json"
"errors"
"fmt"
"net"
"sort"
"openflux/share"
"openflux/transport"
)
// ShareQRPNG renders link as a size x size QR code PNG for the app to show.
@@ -14,9 +16,24 @@ func ShareQRPNG(link string, size int) ([]byte, error) {
return share.PNG(link, size)
}
// ReadShareLink reads an openflux:// link with the core's parser and
// returns share.Result as JSON, the same answer the CLI's --parse-link and
// the iOS library give: {"config":...,"context":...} or
// {"error":...,"code":...,"param":...}. The app words the code itself.
func ReadShareLink(link string) string {
return share.Read(link).JSON()
}
// MakeShareLink builds the link for a share.Config JSON the way every
// client exports one (share.Make) and returns share.Result as JSON:
// {"link":...,"config":...,"context":...} or the error.
func MakeShareLink(configJSON string) string {
return share.MakeJSON(configJSON).JSON()
}
// ParseShareLink decodes a scanned or opened openflux:// link and returns
// its configuration as JSON (share.Config) for the app to turn into a
// profile.
// profile. ReadShareLink says why a link is rejected.
func ParseShareLink(link string) (string, error) {
c, err := share.Decode(link)
if err != nil {
@@ -67,17 +84,23 @@ func ExitShareLink(host, name string) (string, error) {
}
c.Transports = append(c.Transports, t)
}
return share.Encode(c)
return share.MakeLink(c)
}
// exitShareClassic describes a classic single-transport exit to clients.
// The link stays classic (older clients read it too; updated ones upgrade
// to a Session on their own), except for direct, which a link can only
// carry as a Session.
func exitShareClassic(transportType, documentURL, secret, codec string) *share.Config {
c := &share.Config{
Secret: secret,
Context: classicContext(documentURL),
Transports: []share.Transport{{Type: transportType, URL: documentURL}},
context, _ := transport.KDFContexts("", documentURL, []transport.ContextSource{{Type: transportType, URL: documentURL, Priority: 100}})
t := share.Transport{Type: transportType, URL: documentURL}
c := &share.Config{Secret: secret, Context: context}
if transportType == "direct" {
t = share.Transport{Type: "direct", Dial: documentURL}
c.Negotiate = true
}
if codec == "legacy" {
c.Transports = []share.Transport{t}
if codec == transport.CodecLegacy {
c.Codec = codec
}
return c
@@ -87,7 +110,7 @@ func exitShareClassic(transportType, documentURL, secret, codec string) *share.C
// transports and context; direct keeps the listen address, whose host is
// replaced in ExitShareLink. MAX is left out (per-account token).
func exitShareSession(specsJSON, secret string) *share.Config {
specs, context, err := parseSessionSpecs(specsJSON)
specs, context, _, err := parseSessionSpecs(specsJSON)
if err != nil {
return nil
}
@@ -114,12 +137,43 @@ func exitShareSession(specsJSON, secret string) *share.Config {
return c
}
// classicContext is the classic mode's encryption context, as the core
// derives it for --transport with --url: the document URL, or "http://#"
// when there is none (oneme, direct).
func classicContext(documentURL string) string {
if documentURL == "" {
return placeholderURL
// ShareSessionSpecs turns an openflux:// link into the profile StartSession
// (and StartSessionProxy / StartSessionExit) takes: {"context":...,
// "transports":[{name,type,url,priority,params}]}, with the link's own
// context and carrier names, so an app does not have to interpret the link
// itself (and cannot drift from the other clients doing so). A one-carrier
// link works there too: the Session speaks classic to a classic node.
func ShareSessionSpecs(link string) (string, error) {
c, err := share.Decode(link)
if err != nil {
return "", err
}
return documentURL
return sessionSpecsOf(c), nil
}
// sessionSpecsOf is ShareSessionSpecs for a link already read.
func sessionSpecsOf(c share.Config) string {
specs := make([]sessionSpec, 0, len(c.Transports))
seen := make(map[string]int)
for _, t := range c.Transports {
name := t.Name
if name == "" {
seen[t.Type]++
name = t.Type
if n := seen[t.Type]; n > 1 {
name = fmt.Sprintf("%s-%d", t.Type, n)
}
}
spec := sessionSpec{Name: name, Type: t.Type, URL: t.URL, Priority: t.Priority}
if t.Type == "direct" {
spec.URL = ""
spec.Params = map[string]interface{}{"dial": t.Dial}
}
specs = append(specs, spec)
}
b, _ := json.Marshal(struct {
Context string `json:"context"`
Transports []sessionSpec `json:"transports"`
}{share.ContextOf(c), specs})
return string(b)
}
mobile/share_test.go
+21
View File
@@ -105,3 +105,24 @@ func TestExitShareLinkCarriesCreatedCupsRooms(t *testing.T) {
t.Fatalf("transports %+v", c.Transports)
}
}
// Android reads and makes links with the same answer as the CLI and iOS:
// share.Result, byte for byte.
func TestReadMakeShareLinkAnswerLikeShare(t *testing.T) {
cfg := `{"negotiate":true,"secret":"a shared secret of 32 characters","codec":"batched",` +
`"transports":[{"type":"yandex","url":"https://disk.yandex.ru/i/abc","priority":100},{"type":"direct","dial":"203.0.113.7:9443"}]}`
made := MakeShareLink(cfg)
if made != share.MakeJSON(cfg).JSON() {
t.Fatalf("MakeShareLink %s", made)
}
link := share.MakeJSON(cfg).Link
for _, in := range []string{link, " " + link[:30] + "\n" + link[30:], "https://example.com"} {
if got := ReadShareLink(in); got != share.Read(in).JSON() {
t.Errorf("ReadShareLink(%q) = %s", in, got)
}
}
node, err := NodeShareLink("node", "https://docs.yandex.ru/edit/d/AbC", "a shared secret of 32 characters", "203.0.113.7", 9443)
if want := share.Make(share.NodeConfig("node", "https://docs.yandex.ru/edit/d/AbC", "a shared secret of 32 characters", "203.0.113.7:9443")).Link; err != nil || node != want {
t.Errorf("NodeShareLink %q (%v), want %q", node, err, want)
}
}
node_wizard_test.go
+9 -2
View File
@@ -16,6 +16,7 @@ import (
"openflux/provision"
"openflux/share"
"openflux/transport"
"openflux/transport/yandex"
)
@@ -230,7 +231,11 @@ func TestNodeConfMatchesShareLink(t *testing.T) {
t.Fatal(err)
}
c, _ := share.Decode(link)
if got := pickSessionContext("", conf.Interface["URL"], specs); got != c.Context {
sources := make([]transport.ContextSource, len(specs))
for i, s := range specs {
sources[i] = transport.ContextSource{Type: s.Type, URL: s.URL, Priority: s.Priority}
}
if got, _ := transport.KDFContexts("", conf.Interface["URL"], sources); got != c.Context {
t.Fatalf("%+v: node derives context %q, link says %q\n%s", ts, got, c.Context, out)
}
if len(specs) != len(c.Transports) {
@@ -238,7 +243,9 @@ func TestNodeConfMatchesShareLink(t *testing.T) {
}
for i, s := range specs {
lt := c.Transports[i]
if s.Type != lt.Type || s.Priority != lt.Priority || (s.Type != "direct" && s.URL != lt.URL) {
// share.Make drops the priority of a lone carrier: nothing to rank.
samePriority := s.Priority == lt.Priority || len(c.Transports) == 1 && lt.Priority == 0
if s.Type != lt.Type || !samePriority || (s.Type != "direct" && s.URL != lt.URL) {
t.Fatalf("%+v: carrier %d: node %+v, link %+v", ts, i, s, lt)
}
}
provision/transports.go
+9 -16
View File
@@ -10,6 +10,7 @@ import (
"strings"
"openflux/share"
"openflux/transport"
)
// ChannelTransport is one of a channel's carriers besides direct, which
@@ -28,10 +29,6 @@ var transportPriority = map[string]int{"vyandex": 100, "mailru": 90, "cupsonline
const directPriority = 50
// sessionContextNone is the core's context for a node with no document
// (pickSessionContext's placeholder).
const sessionContextNone = "http://#"
var (
volgaDocURL = regexp.MustCompile(`^https://(docs|disk)\.yandex\.(ru|com|by|kz|uz)/edit/d/[A-Za-z0-9_-]{16,200}$`)
mailruDocURL = regexp.MustCompile(`^https://cloud\.mail\.ru/public/[A-Za-z0-9_-]{2,64}/[A-Za-z0-9_-]{2,128}$`)
@@ -95,20 +92,16 @@ func TransportURL(ts []ChannelTransport, typ string) string {
}
// SessionContext is the encryption context both sides of the channel
// derive: the highest-priority document, cups.online aside (its rooms are
// no document), as the core's pickSessionContext and node-install.sh's
// session_context pick it.
// derive, by the core's one rule (transport.KDFContexts): the
// highest-priority document, cups.online aside (its rooms are no
// document). node-install.sh's session_context picks the same.
func SessionContext(ts []ChannelTransport) string {
best, prio := sessionContextNone, -1
sources := make([]transport.ContextSource, 0, len(ts))
for _, t := range ts {
if t.Type == "cupsonline" || t.URL == "" {
continue
}
if p := transportPriority[t.Type]; p > prio {
best, prio = t.URL, p
}
sources = append(sources, transport.ContextSource{Type: t.Type, URL: t.URL, Priority: transportPriority[t.Type]})
}
return best
context, _ := transport.KDFContexts("", "", sources)
return context
}
// ShareLink is the openflux:// link of a new channel: its carriers in
@@ -129,7 +122,7 @@ func ShareLink(name, key, host string, port int, ts []ChannelTransport) (string,
c.Transports = append(c.Transports, share.Transport{
Type: "direct", Dial: net.JoinHostPort(host, strconv.Itoa(port)), Priority: directPriority,
})
return share.Encode(c)
return share.MakeLink(c)
}
// transportLines is the carriers' part of node-install.sh's config.
share/result.go
+151
View File
@@ -0,0 +1,151 @@
package share
import (
"encoding/json"
"errors"
"strings"
"openflux/transport"
)
// Result is what every entry point answers about a link, as the same JSON
// (the CLI's --parse-link and --make-link, package mobile for Android, the
// iOS library): the configuration and its context, or the link built, or
// why it could not be. Apps map Code (and Param) to their own words; Error
// is the English detail, for logs.
type Result struct {
Config *Config `json:"config,omitempty"`
// Context is the encryption context a client of this link derives.
Context string `json:"context,omitempty"`
Link string `json:"link,omitempty"`
Error string `json:"error,omitempty"`
Code string `json:"code,omitempty"`
Param string `json:"param,omitempty"`
}
// JSON is r as the apps receive it.
func (r Result) JSON() string {
b, err := json.Marshal(r)
if err != nil {
return `{"error":"marshal failed"}`
}
return string(b)
}
// Failed is the Result for err: its Code and Param when it is a link
// problem, just the text otherwise.
func Failed(err error) Result {
r := Result{Error: err.Error()}
var e *Error
if errors.As(err, &e) {
r.Code, r.Param = e.Code, e.Param
}
return r
}
// Read parses a link the way every client does and gives the context a
// client derives for it.
func Read(link string) Result {
c, err := Decode(link)
if err != nil {
return Failed(err)
}
return Result{Config: &c, Context: ContextOf(c)}
}
// Make builds the link for c the way every client exports one, so the
// same configuration gives the same link on every client:
//
// - addresses lose surrounding spaces;
// - the default codec ("batched") is left out, and so is the priority
// of a lone carrier (there is nothing to rank it against);
// - an encrypted link always names its context: the one given, else the
// one both peers derive (transport.KDFContexts); a link without a
// secret carries none, there is nothing to derive.
//
// Result.Config is the configuration as it went into the link.
func Make(c Config) Result {
c.Name = strings.TrimSpace(c.Name)
c.Context = strings.TrimSpace(c.Context)
if c.Codec == "batched" {
c.Codec = ""
}
ts := make([]Transport, len(c.Transports))
for i, t := range c.Transports {
t.Type = strings.TrimSpace(t.Type)
t.Name = strings.TrimSpace(t.Name)
t.URL = strings.TrimSpace(t.URL)
t.Dial = strings.TrimSpace(t.Dial)
if t.Name == t.Type {
t.Name = ""
}
ts[i] = t
}
if len(ts) == 1 {
ts[0].Priority = 0
}
c.Transports = ts
switch {
case c.Secret == "":
c.Context = ""
case c.Context == "":
c.Context = ContextOf(c)
}
link, err := Encode(c)
if err != nil {
return Failed(err)
}
return Result{Config: &c, Context: c.Context, Link: link}
}
// MakeJSON is Make for a configuration given as JSON (share.Config).
func MakeJSON(configJSON string) Result {
var c Config
if err := json.Unmarshal([]byte(configJSON), &c); err != nil {
return Failed(linkError(CodeBadConfig, "", "share: bad configuration JSON", err))
}
return Make(c)
}
// ContextOf is the context a client derives for c: the link's own, else
// the one rule every peer uses.
func ContextOf(c Config) string {
sources := make([]transport.ContextSource, len(c.Transports))
for i, t := range c.Transports {
sources[i] = transport.ContextSource{Type: t.Type, URL: t.URL, Priority: t.Priority}
}
context, _ := transport.KDFContexts(c.Context, "", sources)
return context
}
// NodeConfig is the configuration of a channel the node wizard sets up
// (desktop and Android build it here, so their links are the same): the
// Yandex document first, direct to dial (host:port) as the backup, the
// document as the context, and the channel key.
func NodeConfig(name, documentURL, key, dial string) Config {
return Config{
Name: name,
Negotiate: true,
Secret: key,
Context: documentURL,
Transports: []Transport{
{Type: "vyandex", URL: documentURL, Priority: 100},
{Type: "direct", Dial: dial, Priority: 50},
},
}
}
// MakeLink is Make returning just the link, or the error behind the
// Result (a *Error for a link problem).
func MakeLink(c Config) (string, error) {
r := Make(c)
if r.Error != "" {
return "", r.err()
}
return r.Link, nil
}
// err turns a failed Result back into an error.
func (r Result) err() error {
return &Error{Code: r.Code, Param: r.Param, text: r.Error}
}
share/result_test.go
+151
View File
@@ -0,0 +1,151 @@
package share
import (
"bytes"
"compress/flate"
"encoding/base64"
"encoding/json"
"reflect"
"strings"
"testing"
)
// packed wraps raw bytes the way a link carries its JSON.
func packed(t *testing.T, raw []byte) string {
t.Helper()
var buf bytes.Buffer
w, _ := flate.NewWriter(&buf, flate.BestCompression)
w.Write(raw)
w.Close()
return Prefix + base64.RawURLEncoding.EncodeToString(buf.Bytes())
}
// Every way a link can fail has its code: the apps word the problem from
// it, so a new failure without one would reach users as raw English.
func TestReadCodes(t *testing.T) {
good, err := Encode(sample())
if err != nil {
t.Fatal(err)
}
big := `{"name":"` + strings.Repeat("x", maxPayload) + `","transports":[{"type":"yandex","url":"u"}]}`
for _, tc := range []struct{ name, link, code string }{
{"not a link", "https://example.com", CodeNotLink},
{"other version", "openflux://v2/abc", CodeUnsupportedVersion},
{"case changed", strings.ToUpper(good), CodeCaseChanged},
{"not base64", Prefix + "!!!", CodeDamaged},
{"cut short", good[:len(good)-12], CodeDamaged},
{"too large", packed(t, []byte(big)), CodeTooLarge},
{"not a config", packed(t, []byte("[1,2]")), CodeBadPayload},
} {
r := Read(tc.link)
if r.Code != tc.code || r.Error == "" || r.Config != nil {
t.Errorf("%s: %+v, want code %s", tc.name, r, tc.code)
}
}
if r := Read(good); r.Error != "" || r.Config == nil || r.Context != sample().Context {
t.Fatalf("good link: %+v", r)
}
}
func TestMakeCodes(t *testing.T) {
doc := Transport{Type: "yandex", URL: "https://disk.yandex.ru/i/abc"}
key := "a shared secret of 32 characters"
for _, tc := range []struct {
name string
c Config
code, param string
}{
{"no transports", Config{}, CodeNoTransports, ""},
{"several, classic", Config{Transports: []Transport{doc, doc}}, CodeNeedsSession, ""},
{"session, short key", Config{Negotiate: true, Secret: "short", Transports: []Transport{doc}}, CodeSessionSecret, "16"},
{"classic, short key", Config{Secret: "short", Transports: []Transport{doc}}, CodeShortSecret, "16"},
{"codec", Config{Codec: "zip", Transports: []Transport{doc}}, CodeUnknownCodec, "zip"},
{"MAX", Config{Transports: []Transport{{Type: "oneme"}}}, CodeNotShareable, "oneme"},
{"unknown", Config{Transports: []Transport{{Type: "carrier-pigeon"}}}, CodeUnknownTransport, "carrier-pigeon"},
{"direct, no address", Config{Negotiate: true, Secret: key, Transports: []Transport{{Type: "direct"}}}, CodeDirectNoDial, ""},
{"direct, classic", Config{Secret: key, Transports: []Transport{{Type: "direct", Dial: "203.0.113.7:1"}}}, CodeDirectNeedsSession, ""},
} {
r := Make(tc.c)
if r.Code != tc.code || r.Param != tc.param || r.Link != "" {
t.Errorf("%s: %+v, want %s/%q", tc.name, r, tc.code, tc.param)
}
}
if r := MakeJSON("{"); r.Code != CodeBadConfig {
t.Errorf("bad JSON: %+v", r)
}
if _, err := MakeLink(Config{}); err == nil || err.(*Error).Code != CodeNoTransports {
t.Errorf("MakeLink error %v", err)
}
}
// One configuration, however an app spells it, gives one link, and that
// link reads back as the configuration that went in.
func TestMakeIsCanonical(t *testing.T) {
const docURL = "https://docs.yandex.ru/edit/d/AbC"
key := "a shared secret of 32 characters"
base := Config{Name: "node", Negotiate: true, Secret: key, Transports: []Transport{
{Type: "vyandex", URL: docURL, Priority: 100},
{Type: "direct", Dial: "203.0.113.7:9443", Priority: 50},
}}
want := Make(base)
if want.Error != "" || want.Config.Context != docURL {
t.Fatalf("context not filled by the rule: %+v", want)
}
spelled := base
spelled.Codec = "batched"
spelled.Context = " " + docURL + " "
spelled.Transports = []Transport{
{Type: "vyandex", Name: "vyandex", URL: docURL + " ", Priority: 100},
{Type: " direct", Dial: " 203.0.113.7:9443", Priority: 50},
}
if got := Make(spelled); got.Link != want.Link {
t.Errorf("spelled differently, linked differently:\n%s\n%s", got.Link, want.Link)
}
// A lone carrier is not ranked: apps that keep a priority and apps that
// do not export the same link.
lone := Config{Secret: key, Transports: []Transport{{Type: "mailru", URL: "https://cloud.mail.ru/public/x", Priority: 100}}}
unranked := lone
unranked.Transports = []Transport{{Type: "mailru", URL: "https://cloud.mail.ru/public/x"}}
if a, b := Make(lone), Make(unranked); a.Link != b.Link || a.Config.Transports[0].Priority != 0 {
t.Errorf("lone carrier: %+v vs %+v", a, b)
}
back := Read(want.Link)
if !reflect.DeepEqual(back.Config, want.Config) || back.Context != want.Context {
t.Errorf("read back %+v, made %+v", back.Config, want.Config)
}
for _, tc := range []struct {
name string
c Config
context string
}{
{"classic document with a key", Config{Secret: key, Transports: []Transport{{Type: "mailru", URL: "https://cloud.mail.ru/public/x"}}}, "https://cloud.mail.ru/public/x"},
{"cups rooms", Config{Secret: key, Transports: []Transport{{Type: "cupsonline", URL: "WyJhIl0"}}}, "http://#"},
{"direct only", Config{Negotiate: true, Secret: key, Transports: []Transport{{Type: "direct", Dial: "203.0.113.7:1"}}}, "http://#"},
{"explicit kept", Config{Negotiate: true, Secret: key, Context: "https://x/y", Transports: []Transport{{Type: "yandex", URL: "https://z"}}}, "https://x/y"},
{"no key, no context", Config{Context: "https://x/y", Transports: []Transport{{Type: "yandex", URL: "https://z"}}}, ""},
} {
r := Make(tc.c)
if r.Error != "" || r.Config.Context != tc.context || r.Context != tc.context {
t.Errorf("%s: %+v, want context %q", tc.name, r, tc.context)
}
}
}
// The JSON the apps parse: field names are the contract.
func TestResultJSON(t *testing.T) {
var fail map[string]string
json.Unmarshal([]byte(Read("nope").JSON()), &fail)
if fail["code"] != CodeNotLink || fail["error"] == "" {
t.Errorf("failure JSON %v", fail)
}
var ok struct {
Config *Config `json:"config"`
Context string `json:"context"`
Link string `json:"link"`
}
json.Unmarshal([]byte(Make(sample()).JSON()), &ok)
if ok.Config == nil || ok.Link == "" || ok.Context != sample().Context {
t.Errorf("success JSON %+v", ok)
}
}
share/share.go
+100 -19
View File
@@ -12,12 +12,14 @@ import (
"compress/flate"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"io"
"strconv"
"strings"
qrcode "github.com/skip2/go-qrcode"
"openflux/utils"
)
// Prefix starts every link; the path segment is the format version.
@@ -27,6 +29,51 @@ const Prefix = "openflux://v1/"
// the decoder allocate without limit.
const maxPayload = 16 << 10
// Why a link cannot be read or made (Error.Code, Result.Code). The codes
// are the contract with the apps: they word each one for their users, the
// core only decides which it is.
const (
CodeNotLink = "not_link"
CodeUnsupportedVersion = "unsupported_version"
CodeCaseChanged = "case_changed"
CodeDamaged = "damaged" // not base64url or not DEFLATE: mangled or cut on the way
CodeTooLarge = "too_large"
CodeBadPayload = "bad_payload" // decompresses, but is not a configuration
CodeBadConfig = "bad_config" // Make: the configuration JSON does not parse
CodeNoTransports = "no_transports"
CodeNeedsSession = "several_need_session"
CodeSessionSecret = "session_secret" // Param: the minimum length
CodeShortSecret = "short_secret" // Param: the minimum length
CodeUnknownCodec = "unknown_codec" // Param: the codec
CodeNotShareable = "not_shareable" // Param: the transport type (MAX)
CodeUnknownTransport = "unknown_transport"
CodeDirectNoDial = "direct_no_dial"
CodeDirectNeedsSession = "direct_needs_session"
)
// Error is a link that cannot be used: Code says which problem it is (for
// the apps), Param the value it is about, Error() the English detail the
// CLI prints.
type Error struct {
Code string
Param string
text string
cause error
}
func (e *Error) Error() string {
if e.cause != nil {
return e.text + ": " + e.cause.Error()
}
return e.text
}
func (e *Error) Unwrap() error { return e.cause }
func linkError(code, param, text string, cause error) error {
return &Error{Code: code, Param: param, text: text, cause: cause}
}
// Transport is one transport the client should run.
type Transport struct {
Type string `json:"type"`
@@ -64,30 +111,37 @@ var knownTypes = map[string]bool{
// Validate reports whether c describes something a client can connect with.
func (c *Config) Validate() error {
if len(c.Transports) == 0 {
return errors.New("share: no transports")
return linkError(CodeNoTransports, "", "share: no transports", nil)
}
if len(c.Transports) > 1 && !c.Negotiate {
return errors.New("share: several transports need a negotiated session")
return linkError(CodeNeedsSession, "", "share: several transports need a negotiated session", nil)
}
if c.Negotiate && len(c.Secret) < 16 {
return errors.New("share: a negotiated session needs a secret of at least 16 characters")
// Characters as Kotlin and Java count them (UTF-16 units), so a link
// is valid or not the same way on every client.
if c.Negotiate && utils.SecretChars(c.Secret) < utils.MinSecretChars {
return linkError(CodeSessionSecret, strconv.Itoa(utils.MinSecretChars),
fmt.Sprintf("share: a negotiated session needs a secret of at least %d characters", utils.MinSecretChars), nil)
}
if c.Secret != "" && len(c.Secret) < 16 {
return errors.New("share: the secret must be at least 16 characters")
if c.Secret != "" && utils.SecretChars(c.Secret) < utils.MinSecretChars {
return linkError(CodeShortSecret, strconv.Itoa(utils.MinSecretChars),
fmt.Sprintf("share: the secret must be at least %d characters", utils.MinSecretChars), nil)
}
if c.Codec != "" && c.Codec != "batched" && c.Codec != "legacy" {
return fmt.Errorf("share: unknown codec %q", c.Codec)
return linkError(CodeUnknownCodec, c.Codec, fmt.Sprintf("share: unknown codec %q", c.Codec), nil)
}
for _, t := range c.Transports {
if !knownTypes[t.Type] {
return fmt.Errorf("share: unknown transport type %q", t.Type)
if t.Type == "oneme" {
return linkError(CodeNotShareable, t.Type, "share: MAX (oneme) cannot be shared: its token belongs to one account", nil)
}
return linkError(CodeUnknownTransport, t.Type, fmt.Sprintf("share: unknown transport type %q", t.Type), nil)
}
if t.Type == "direct" {
if t.Dial == "" {
return errors.New("share: direct needs the exit's address")
return linkError(CodeDirectNoDial, "", "share: direct needs the exit's address", nil)
}
if !c.Negotiate {
return errors.New("share: direct only works in a negotiated session")
return linkError(CodeDirectNeedsSession, "", "share: direct only works in a negotiated session", nil)
}
}
}
@@ -118,28 +172,38 @@ func Encode(c Config) (string, error) {
}
// Decode parses and validates an openflux:// link.
//
// It is lenient about how the link travelled, the same way on every
// client: whitespace and line breaks inside it (a link copied out of a
// terminal or a chat wraps), base64 padding ("=", which some encoders add)
// and the standard base64 alphabet ("+/" for "-_") are all accepted. The
// link Encode writes has none of them.
func Decode(link string) (Config, error) {
link = strings.TrimSpace(link)
if !strings.HasPrefix(link, Prefix) {
if strings.HasPrefix(link, "openflux://") {
return Config{}, errors.New("share: unsupported link version; update OpenFlux")
if strings.HasPrefix(strings.ToLower(link), "openflux://") && !strings.HasPrefix(link, "openflux://") {
return Config{}, linkError(CodeCaseChanged, "", "share: the link's letters changed case on the way (openflux:// links are case-sensitive); copy it again", nil)
}
return Config{}, errors.New("share: not an openflux:// link")
if strings.HasPrefix(link, "openflux://") {
return Config{}, linkError(CodeUnsupportedVersion, "", "share: unsupported link version; update OpenFlux", nil)
}
return Config{}, linkError(CodeNotLink, "", "share: not an openflux:// link", nil)
}
packed, err := base64.RawURLEncoding.DecodeString(strings.TrimPrefix(link, Prefix))
body := normalizeBody(strings.TrimPrefix(link, Prefix))
packed, err := base64.RawURLEncoding.DecodeString(body)
if err != nil {
return Config{}, fmt.Errorf("share: bad link encoding: %w", err)
return Config{}, linkError(CodeDamaged, "", fmt.Sprintf("share: bad link encoding (%d characters after the prefix; truncated or mangled?)", len(body)), err)
}
raw, err := io.ReadAll(io.LimitReader(flate.NewReader(bytes.NewReader(packed)), maxPayload+1))
if err != nil {
return Config{}, fmt.Errorf("share: bad link payload: %w", err)
return Config{}, linkError(CodeDamaged, "", "share: bad link payload (not raw DEFLATE; zlib/gzip-wrapped or truncated?)", err)
}
if len(raw) > maxPayload {
return Config{}, errors.New("share: link payload too large")
return Config{}, linkError(CodeTooLarge, "", "share: link payload too large", nil)
}
var c Config
if err := json.Unmarshal(raw, &c); err != nil {
return Config{}, fmt.Errorf("share: bad link payload: %w", err)
return Config{}, linkError(CodeBadPayload, "", "share: bad link payload", err)
}
if err := c.Validate(); err != nil {
return Config{}, err
@@ -147,6 +211,23 @@ func Decode(link string) (Config, error) {
return c, nil
}
// normalizeBody undoes what copying and other encoders do to the base64url
// part of a link.
func normalizeBody(b string) string {
b = strings.Map(func(r rune) rune {
switch r {
case ' ', '\t', '\r', '\n', '\u00a0', '\u200b':
return -1
case '+':
return '-'
case '/':
return '_'
}
return r
}, b)
return strings.TrimRight(b, "=")
}
func qr(link string) (*qrcode.QRCode, error) {
// Medium correction: survives a slightly glared phone screen while
// keeping the code small enough to scan off another phone.
share/share_test.go
+55
View File
@@ -113,3 +113,58 @@ func deflated(t *testing.T, raw []byte) string {
_ = w.Close()
return base64.RawURLEncoding.EncodeToString(buf.Bytes())
}
// Links reach Decode after copying, chats and other encoders; every client
// must read them the same way.
func TestDecodeTolerant(t *testing.T) {
c := Config{Name: "tt", Secret: "0123456789abcdef0123", Context: "https://cloud.mail.ru/public/AbCd/EfGh",
Transports: []Transport{{Type: "mailru", URL: "https://cloud.mail.ru/public/AbCd/EfGh"}}}
var link string
for {
l, err := Encode(c)
if err != nil {
t.Fatal(err)
}
if len(strings.TrimPrefix(l, Prefix))%4 != 0 {
link = l
break
}
c.Name += "x"
}
body := strings.TrimPrefix(link, Prefix)
raw, err := base64.RawURLEncoding.DecodeString(body)
if err != nil {
t.Fatal(err)
}
variants := map[string]string{
"padded": Prefix + base64.URLEncoding.EncodeToString(raw),
"wrapped": link[:40] + "\n" + link[40:80] + "\r\n " + link[80:],
"std alphabet": Prefix + base64.RawStdEncoding.EncodeToString(raw),
"nbsp": link[:50] + " " + link[50:],
}
for name, v := range variants {
got, err := Decode(v)
if err != nil {
t.Errorf("%s: %v", name, err)
continue
}
if got.Name != c.Name {
t.Errorf("%s: name %q", name, got.Name)
}
}
if _, err := Decode(strings.ToUpper(link)); err == nil || !strings.Contains(err.Error(), "case") {
t.Errorf("upper-cased link: %v", err)
}
}
// The secret is counted in characters as Kotlin counts them, not bytes.
func TestSecretCountedInCharacters(t *testing.T) {
c := Config{Secret: "ключключкл", Transports: []Transport{{Type: "mailru", URL: "https://cloud.mail.ru/public/a/b"}}}
if err := c.Validate(); err == nil {
t.Fatal("10 Cyrillic letters (20 bytes) accepted as a 16-character secret")
}
c.Secret = "ключключключключ"
if err := c.Validate(); err != nil {
t.Fatalf("16 Cyrillic letters: %v", err)
}
}
share_cli.go
+45 -3
View File
@@ -2,6 +2,7 @@ package main
import (
"fmt"
"io"
"log"
"net"
"os"
@@ -67,11 +68,12 @@ func printShare(c share.Config, skipped []string) {
for _, why := range skipped {
log.Printf("--share: left out %s", why)
}
link, err := share.Encode(c)
if err != nil {
log.Printf("--share: %v", err)
r := share.Make(c)
if r.Error != "" {
log.Printf("--share: %s", r.Error)
return
}
link := r.Link
qr, err := share.Terminal(link)
if err != nil {
log.Printf("--share: %v", err)
@@ -79,6 +81,46 @@ func printShare(c share.Config, skipped []string) {
}
log.Printf("Share link for clients (contains the encryption key): %s", link)
fmt.Fprint(os.Stderr, qr)
// The bare link on a line of its own, to copy without the log prefix.
fmt.Fprintln(os.Stderr, link)
}
// runParseLink reads an openflux:// link (arg, or stdin for "-") with the
// core's parser and prints share.Result as JSON: {"config":...,"context":...}
// or {"error":...,"code":...,"param":...}. Apps word the code themselves.
func runParseLink(arg string, stdin io.Reader, stdout io.Writer) int {
link, err := argOrStdin(arg, stdin)
if err != nil {
return writeLinkResult(stdout, share.Failed(err))
}
return writeLinkResult(stdout, share.Read(link))
}
// runMakeLink builds the link for a share.Config JSON (arg, or stdin for
// "-") the way every client exports one and prints share.Result as JSON:
// {"link":...,"config":...,"context":...} or the error.
func runMakeLink(arg string, stdin io.Reader, stdout io.Writer) int {
cfg, err := argOrStdin(arg, stdin)
if err != nil {
return writeLinkResult(stdout, share.Failed(err))
}
return writeLinkResult(stdout, share.MakeJSON(cfg))
}
func argOrStdin(arg string, stdin io.Reader) (string, error) {
if arg != "-" {
return arg, nil
}
b, err := io.ReadAll(io.LimitReader(stdin, 64<<10))
return string(b), err
}
func writeLinkResult(w io.Writer, r share.Result) int {
fmt.Fprintln(w, r.JSON())
if r.Error != "" {
return 1
}
return 0
}
// publicIPv4 guesses the address clients should dial: the first global
share_cli_test.go
+36
View File
@@ -4,6 +4,7 @@ import (
"strings"
"testing"
"openflux/provision"
"openflux/share"
)
@@ -60,3 +61,38 @@ func TestShareConfigCarriesCreatedCupsRooms(t *testing.T) {
t.Fatalf("transports %+v, skipped %v", c.Transports, skipped)
}
}
// --parse-link and --make-link answer with share.Result, byte for byte
// what package mobile and the iOS library answer, and the wizard's link is
// the one share.Make builds for its configuration.
func TestLinkCommandsAnswerLikeShare(t *testing.T) {
cfg := `{"name":"node","negotiate":true,"secret":"a shared secret of 32 characters",` +
`"transports":[{"type":"vyandex","url":"https://docs.yandex.ru/edit/d/AbC","priority":100}]}`
var out strings.Builder
if code := runMakeLink("-", strings.NewReader(cfg), &out); code != 0 {
t.Fatalf("--make-link exit %d: %s", code, out.String())
}
made := share.MakeJSON(cfg)
if out.String() != made.JSON()+"\n" {
t.Fatalf("--make-link %s, share.MakeJSON %s", out.String(), made.JSON())
}
wrapped := made.Link[:40] + "\r\n" + made.Link[40:]
out.Reset()
if code := runParseLink("-", strings.NewReader(wrapped), &out); code != 0 || out.String() != share.Read(made.Link).JSON()+"\n" {
t.Fatalf("--parse-link exit %d: %s", code, out.String())
}
out.Reset()
if code := runParseLink("https://example.com", nil, &out); code != 1 || !strings.Contains(out.String(), `"code":"not_link"`) {
t.Fatalf("--parse-link on garbage: exit %d %s", code, out.String())
}
// The wizard's link for a Yandex-document channel is share.NodeConfig's.
doc := "https://docs.yandex.ru/edit/d/AbCdEfGhIjKlMnOpQrStUv"
link, err := provision.ShareLink("node", "a shared secret of 32 characters", "203.0.113.7", 9443,
[]provision.ChannelTransport{{Type: "vyandex", URL: doc}})
want := share.Make(share.NodeConfig("node", doc, "a shared secret of 32 characters", "203.0.113.7:9443"))
if err != nil || link != want.Link {
t.Fatalf("wizard link %q (%v), share.Make %q", link, err, want.Link)
}
}
transport/codec.go
+324
View File
@@ -0,0 +1,324 @@
package transport
import (
"fmt"
"sync"
"sync/atomic"
"time"
"openflux/utils"
)
// Classic-mode codec names, as --codec and openflux:// links spell them.
const (
CodecBatched = "batched"
CodecLegacy = "legacy"
)
// Codec fallback timing for the side that speaks first (the client): with
// nothing heard from the peer, the other framing is tried codecFirstFallback
// after the first send, then the two alternate every codecFallbackEvery.
// That is faster than the KDF context rotation (see EncryptedTransport), so
// both framings get tried under every candidate context.
const (
codecFirstFallback = 2 * time.Second
codecFallbackEvery = time.Second
)
// CodecTransport is the classic (non-Session) mode's codec. It replaces the
// fixed BatchedTransport / CompressedTransport pair, which had to match on
// both peers: batched against legacy dropped every packet in silence.
//
// The two framings never collide on their first byte:
//
// batch-v2 0x02 many packets per frame, zstd when it helps
// legacy 0x00 / 0x1F one packet per frame, raw / LZ4
//
// so every frame is decoded whichever framing it uses, and the sending
// framing follows the peer:
//
// - once the peer has sent a batch frame (an empty one is the probe the
// iOS fork's adaptive codec sends), frames go out batched;
// - once it has sent only legacy frames, they go out legacy;
// - before anything is heard, the preferred framing is used; the
// initiating side (the client) switches to the other one when the peer
// stays silent, so a peer that decodes only one framing still gets
// through.
//
// Old batched-only or legacy-only peers therefore keep working, and
// updated peers converge on batching.
type CodecTransport struct {
Transport
preferred bool // true: batched
initiator bool
batchNow atomic.Bool // current send framing before the peer is known
peerBatch atomic.Bool
peerLegacy atomic.Bool
heard atomic.Bool
firstSend atomic.Int64
lastSwitch atomic.Int64
queue chan []byte
lingerMs int
maxBatchBytes int
maxBatchCount int
running atomic.Bool
lifecycle sync.Mutex
stopOnce sync.Once
stopCh chan struct{}
mu sync.RWMutex
userCb func([]byte)
sendErrors atomic.Uint64
unknown atomic.Uint64
}
// NewCodecTransport wraps inner. preferred is CodecBatched (also when
// empty) or CodecLegacy; initiator is true on the side that speaks first.
func NewCodecTransport(inner Transport, preferred string, initiator bool) *CodecTransport {
c := &CodecTransport{
Transport: inner,
preferred: preferred != CodecLegacy,
initiator: initiator,
queue: make(chan []byte, batchQueueDepth),
lingerMs: envInt("OPENFLUX_BATCH_LINGER_MS", defaultLingerMs),
maxBatchBytes: min(envInt("OPENFLUX_BATCH_BYTES", defaultMaxBatchBytes), maxFrameBytes-65537),
maxBatchCount: min(envInt("OPENFLUX_BATCH_COUNT", defaultMaxBatchCount), maxFrameRecords-1),
stopCh: make(chan struct{}),
}
c.batchNow.Store(c.preferred)
utils.Debugf("[CODEC] created: preferred=%s initiator=%v (both framings accepted on receive)",
codecName(c.preferred), initiator)
return c
}
func codecName(batch bool) string {
if batch {
return CodecBatched
}
return CodecLegacy
}
// Current names the framing frames are sent in now.
func (c *CodecTransport) Current() string { return codecName(c.sendBatch()) }
func (c *CodecTransport) Start() error {
c.lifecycle.Lock()
defer c.lifecycle.Unlock()
select {
case <-c.stopCh:
return fmt.Errorf("codec transport is stopped")
default:
}
if c.running.Load() {
return nil
}
if err := c.Transport.Start(); err != nil {
return err
}
c.running.Store(true)
go c.flushLoop()
return nil
}
func (c *CodecTransport) Stop() error {
c.lifecycle.Lock()
defer c.lifecycle.Unlock()
select {
case <-c.stopCh:
return nil
default:
}
c.running.Store(false)
c.stopOnce.Do(func() { close(c.stopCh) })
return c.Transport.Stop()
}
// Send copies the packet and queues it. A full queue drops it (TCP
// retransmits, UDP loses the datagram).
func (c *CodecTransport) Send(data []byte) error {
if !c.running.Load() {
return fmt.Errorf("codec transport is not running")
}
if len(data) > 65535 {
return fmt.Errorf("packet too large for a frame: %d bytes", len(data))
}
p := append([]byte(nil), data...)
select {
case c.queue <- p:
return nil
default:
utils.Debugf("[CODEC] Send: QUEUE FULL, dropped %d bytes", len(p))
return fmt.Errorf("codec queue full")
}
}
func (c *CodecTransport) Receive(callback func([]byte)) {
c.mu.Lock()
c.userCb = callback
c.mu.Unlock()
c.Transport.Receive(c.receive)
}
func (c *CodecTransport) receive(data []byte) {
if len(data) == 0 {
return
}
var pkts [][]byte
switch data[0] {
case batchFormatVersion:
p, err := decodeBatch(data)
if err != nil {
utils.Debugf("[CODEC] batch frame (%d bytes) undecodable: %v", len(data), err)
return
}
pkts = p
if !c.peerBatch.Swap(true) {
utils.Debugf("[CODEC] peer sends batch-v2 frames: sending batched from now on")
}
case 0x00, CompressionMarker:
if len(data) == 1 {
// A lone 0x00 is a carrier keepalive (Volga sends one through
// its relay), not a frame of the peer's codec.
return
}
p, err := decompress(data)
if err != nil {
utils.Debugf("[CODEC] legacy frame (%d bytes) undecodable: %v", len(data), err)
return
}
pkts = [][]byte{p}
if !c.peerLegacy.Swap(true) && !c.peerBatch.Load() {
utils.Debugf("[CODEC] peer sends legacy frames: sending legacy until it sends a batch frame")
}
default:
n := c.unknown.Add(1)
if utils.Throttled("codec.unknown", 30*time.Second) {
utils.Infof("[CODEC] %d frame(s) from the peer in no classic framing (first bytes % x)%s",
n, data[:min(len(data), 4)], layeringHint(data))
}
return
}
if !c.heard.Swap(true) {
mode := c.Current()
if c.initiator && mode != codecName(c.preferred) {
utils.Infof("[CODEC] peer answered in %s framing (preferred was %s): staying on it", mode, codecName(c.preferred))
} else {
utils.Debugf("[CODEC] peer heard; sending %s", mode)
}
}
c.mu.RLock()
cb := c.userCb
c.mu.RUnlock()
if cb == nil {
return
}
for _, p := range pkts {
if len(p) > 0 {
cb(p)
}
}
}
// layeringHint explains a frame that is neither framing.
func layeringHint(p []byte) string {
if len(p) >= 3 && p[0] == encryptedMagic[0] && p[1] == encryptedMagic[1] && p[2] == encryptedMagic[2] {
return ": an encrypted record outside any codec, i.e. the peer runs a Session (--negotiate / --transports / .conf / a Session profile)"
}
if p[0]>>4 == 4 {
return ": a bare IPv4 packet (a peer without any codec)"
}
return ""
}
// sendBatch picks the framing for the next frame.
func (c *CodecTransport) sendBatch() bool {
if c.peerBatch.Load() {
return true
}
if c.peerLegacy.Load() {
return false
}
if !c.initiator || c.heard.Load() {
return c.batchNow.Load()
}
now := time.Now().UnixNano()
first := c.firstSend.Load()
if first == 0 {
c.firstSend.CompareAndSwap(0, now)
c.lastSwitch.Store(now)
return c.batchNow.Load()
}
last := c.lastSwitch.Load()
wait := codecFallbackEvery
if last == first {
wait = codecFirstFallback
}
if time.Duration(now-last) >= wait && c.lastSwitch.CompareAndSwap(last, now) {
was := c.batchNow.Load()
c.batchNow.Store(!was)
if utils.Throttled("codec.fallback", 10*time.Second) {
utils.Infof("[CODEC] no answer from the peer in %s framing; trying %s", codecName(was), codecName(!was))
}
}
return c.batchNow.Load()
}
func (c *CodecTransport) flushLoop() {
for c.running.Load() {
var first []byte
select {
case <-c.stopCh:
return
case first = <-c.queue:
}
if !c.sendBatch() {
c.write(compress(first))
continue
}
batch := [][]byte{first}
size := 2 + len(first)
drain:
for size < c.maxBatchBytes && len(batch) < c.maxBatchCount {
select {
case p := <-c.queue:
batch = append(batch, p)
size += 2 + len(p)
default:
break drain
}
}
if c.lingerMs > 0 && size < c.maxBatchBytes && len(batch) < c.maxBatchCount {
timer := time.NewTimer(time.Duration(c.lingerMs) * time.Millisecond)
linger:
for size < c.maxBatchBytes && len(batch) < c.maxBatchCount {
select {
case <-c.stopCh:
timer.Stop()
return
case p := <-c.queue:
batch = append(batch, p)
size += 2 + len(p)
case <-timer.C:
break linger
}
}
timer.Stop()
}
c.write(encodeBatch(batch))
}
}
func (c *CodecTransport) write(frame []byte) {
if err := c.Transport.Send(frame); err != nil {
n := c.sendErrors.Add(1)
utils.Debugf("[CODEC] send error (total=%d): %v", n, err)
}
}
// SendErrors counts Transport.Send failures seen by the codec.
func (c *CodecTransport) SendErrors() uint64 { return c.sendErrors.Load() }
transport/compat_test.go
+353
View File
@@ -0,0 +1,353 @@
package transport
import (
"bytes"
"sync"
"sync/atomic"
"testing"
"time"
"openflux/transport/control"
)
// Compatibility between the layerings, codecs and KDF contexts that peers
// of different builds use. "old" stacks are built exactly as the core did
// before (Encrypted(Batched|Compressed(carrier)) with a fixed context);
// "new" ones as it does now.
// asyncWire is one end of an in-memory carrier that delivers in order on
// its own goroutine, like a real one (no re-entrant Send -> Receive).
type asyncWire struct {
mu sync.Mutex
cb func([]byte)
out chan []byte
down atomic.Bool
}
func asyncPair() (*asyncWire, *asyncWire) {
a := &asyncWire{out: make(chan []byte, 4096)}
b := &asyncWire{out: make(chan []byte, 4096)}
pump := func(from, to *asyncWire) {
for p := range from.out {
to.mu.Lock()
cb := to.cb
to.mu.Unlock()
if cb != nil {
cb(p)
}
}
}
go pump(a, b)
go pump(b, a)
return a, b
}
func (w *asyncWire) Start() error { return nil }
func (w *asyncWire) Stop() error { return nil }
func (w *asyncWire) IsConnected() bool { return true }
func (w *asyncWire) Stats() TransportStats { return TransportStats{} }
func (w *asyncWire) Receive(cb func([]byte)) {
w.mu.Lock()
w.cb = cb
w.mu.Unlock()
}
func (w *asyncWire) Send(p []byte) error {
if !w.down.Load() {
w.out <- append([]byte(nil), p...)
}
return nil
}
type peer interface {
Start() error
Stop() error
Send([]byte) error
Receive(func([]byte))
}
const compatSecret = "compat test secret 0123456789"
func oldClassic(t *testing.T, w Transport, codec, context string, exit bool) peer {
t.Helper()
var inner Transport = w
if codec == CodecLegacy {
inner = NewCompressedTransport(inner)
} else {
inner = NewBatchedTransport(inner)
}
enc, err := NewEncryptedTransport(inner, compatSecret, context, exit)
if err != nil {
t.Fatal(err)
}
return enc
}
func newClassic(t *testing.T, w Transport, codec, context string, alternates []string, exit bool) peer {
t.Helper()
enc, err := NewEncryptedTransport(NewCodecTransport(w, codec, !exit), compatSecret, context, exit)
if err != nil {
t.Fatal(err)
}
enc.SetAlternateContexts(alternates)
return enc
}
func compatSession(t *testing.T, w Transport, context string, alternates []string, exit, classic bool, codec string) *Session {
t.Helper()
s, err := NewSession(PeerParameters{
Capabilities: control.CapabilityIPv4 | control.CapabilityTCP | control.CapabilityUDP,
MaxPacketSize: MaxNegotiatedPacket,
}, exit)
if err != nil {
t.Fatal(err)
}
if classic {
s.SetClassic(codec)
}
s.SetAlternateContexts(alternates)
if err := s.AddTransport("carrier", w, compatSecret, context, 100); err != nil {
t.Fatal(err)
}
return s
}
// roundTrip starts both ends, has the client send an IPv4 packet every
// 50ms and the exit echo each one, and reports how long the first echo
// took (or fails after limit).
func roundTrip(t *testing.T, client, exit peer, limit time.Duration, while ...func()) time.Duration {
t.Helper()
exit.Receive(func(p []byte) { _ = exit.Send(p) })
got := make(chan struct{}, 1)
client.Receive(func(p []byte) {
if len(p) >= 20 && p[0]>>4 == 4 {
select {
case got <- struct{}{}:
default:
}
}
})
if err := exit.Start(); err != nil {
t.Fatalf("exit start: %v", err)
}
began := time.Now()
if err := client.Start(); err != nil {
t.Fatalf("client start: %v", err)
}
defer client.Stop()
defer exit.Stop()
tick := time.NewTicker(50 * time.Millisecond)
defer tick.Stop()
deadline := time.After(limit)
for {
_ = client.Send(testIPv4(60, 6))
select {
case <-got:
d := time.Since(began)
for _, f := range while {
f()
}
return d
case <-deadline:
t.Fatalf("no echo within %v", limit)
return 0
case <-tick.C:
}
}
}
func TestCodecFallbackAgainstOldPeers(t *testing.T) {
cases := []struct {
name string
client, exit func(*asyncWire, *asyncWire) (peer, peer)
}{
{"new batched client -> old legacy exit", func(a, b *asyncWire) (peer, peer) {
return newClassic(t, a, CodecBatched, "ctx", nil, false), oldClassic(t, b, CodecLegacy, "ctx", true)
}, nil},
{"new legacy client -> old batched exit", func(a, b *asyncWire) (peer, peer) {
return newClassic(t, a, CodecLegacy, "ctx", nil, false), oldClassic(t, b, CodecBatched, "ctx", true)
}, nil},
{"old legacy client -> new batched exit", func(a, b *asyncWire) (peer, peer) {
return oldClassic(t, a, CodecLegacy, "ctx", false), newClassic(t, b, CodecBatched, "ctx", nil, true)
}, nil},
{"old batched client -> new legacy exit", func(a, b *asyncWire) (peer, peer) {
return oldClassic(t, a, CodecBatched, "ctx", false), newClassic(t, b, CodecLegacy, "ctx", nil, true)
}, nil},
{"new legacy client -> new batched exit", func(a, b *asyncWire) (peer, peer) {
return newClassic(t, a, CodecLegacy, "ctx", nil, false), newClassic(t, b, CodecBatched, "ctx", nil, true)
}, nil},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
a, b := asyncPair()
cl, ex := c.client(a, b)
d := roundTrip(t, cl, ex, 8*time.Second)
t.Logf("first echo after %v", d.Round(10*time.Millisecond))
})
}
}
// An iOS-fork peer's empty-batch probe makes the codec send batched.
func TestCodecFollowsBatchProbe(t *testing.T) {
w := &fakeTransport{}
c := NewCodecTransport(w, CodecLegacy, false)
c.Receive(func([]byte) {})
deliver := func(p []byte) {
w.mu.Lock()
cb := w.cb
w.mu.Unlock()
cb(p)
}
deliver([]byte{0x00}) // a Volga carrier keepalive: ignored
if c.heard.Load() {
t.Fatal("a lone 0x00 counted as the peer's codec")
}
deliver(encodeBatch(nil))
if c.Current() != CodecBatched {
t.Fatalf("after a batch probe the codec sends %s", c.Current())
}
}
func TestContextFallback(t *testing.T) {
t.Run("new client rotates to an old exit's context", func(t *testing.T) {
a, b := asyncPair()
cl := newClassic(t, a, CodecBatched, "https://docs/d", []string{ContextPlaceholder, "rooms"}, false)
ex := oldClassic(t, b, CodecBatched, "rooms", true)
d := roundTrip(t, cl, ex, 15*time.Second)
t.Logf("first echo after %v", d.Round(10*time.Millisecond))
})
t.Run("new exit answers an old client under its context", func(t *testing.T) {
a, b := asyncPair()
cl := oldClassic(t, a, CodecBatched, "rooms", false)
ex := newClassic(t, b, CodecBatched, ContextPlaceholder, []string{"rooms"}, true)
d := roundTrip(t, cl, ex, 5*time.Second)
if d > 2*time.Second {
t.Fatalf("exit took %v to answer under the client's context", d)
}
})
t.Run("Session over mismatched contexts", func(t *testing.T) {
a, b := asyncPair()
cl := compatSession(t, a, "https://docs/d", []string{ContextPlaceholder}, false, false, "")
ex := compatSession(t, b, ContextPlaceholder, []string{"https://docs/d"}, true, false, "")
roundTrip(t, cl, ex, 5*time.Second)
})
}
func TestSessionClassicCompat(t *testing.T) {
t.Run("new client falls back to an old classic exit", func(t *testing.T) {
a, b := asyncPair()
cl := compatSession(t, a, "ctx", nil, false, true, CodecBatched)
ex := oldClassic(t, b, CodecLegacy, "ctx", true)
d := roundTrip(t, cl, ex, 12*time.Second, func() {
if m := cl.Mode(); m != "classic" {
t.Errorf("client mode %q", m)
}
})
t.Logf("first echo after %v (classic)", d.Round(10*time.Millisecond))
})
t.Run("old classic client served by a classic-configured exit", func(t *testing.T) {
a, b := asyncPair()
cl := oldClassic(t, a, CodecBatched, "ctx", false)
ex := compatSession(t, b, "ctx", nil, true, true, CodecBatched)
roundTrip(t, cl, ex, 5*time.Second, func() {
if m := ex.Mode(); m != "classic" {
t.Errorf("exit mode %q", m)
}
})
})
t.Run("new classic profile upgrades to a Session", func(t *testing.T) {
a, b := asyncPair()
cl := compatSession(t, a, "ctx", nil, false, true, CodecLegacy)
ex := compatSession(t, b, "ctx", nil, true, true, CodecBatched)
roundTrip(t, cl, ex, 5*time.Second, func() {
deadline := time.Now().Add(3 * time.Second)
for cl.Mode() != "session" && time.Now().Before(deadline) {
time.Sleep(20 * time.Millisecond)
}
if m := cl.Mode(); m != "session" {
t.Errorf("client mode %q", m)
}
})
})
t.Run("old Session client with a classic-configured exit", func(t *testing.T) {
a, b := asyncPair()
cl := compatSession(t, a, "ctx", nil, false, false, "")
ex := compatSession(t, b, "ctx", nil, true, true, CodecBatched)
roundTrip(t, cl, ex, 5*time.Second)
})
t.Run("a Session-only exit does not serve classic clients", func(t *testing.T) {
a, b := asyncPair()
cl := oldClassic(t, a, CodecBatched, "ctx", false)
ex := compatSession(t, b, "ctx", nil, true, false, "")
var got atomic.Int64
ex.Receive(func([]byte) { got.Add(1) })
_ = ex.Start()
_ = cl.Start()
defer cl.Stop()
defer ex.Stop()
for i := 0; i < 20; i++ {
_ = cl.Send(testIPv4(60, 6))
time.Sleep(20 * time.Millisecond)
}
if got.Load() != 0 {
t.Fatalf("strict exit delivered %d classic packets", got.Load())
}
})
}
// A Session exit prefers its Session client over a classic one: classic
// frames on the carrier (a replayed capture, another device) must not
// take the replies away from it.
func TestSessionClassicYieldsToSession(t *testing.T) {
a, b := asyncPair()
cl := compatSession(t, a, "ctx", nil, false, false, "")
ex := compatSession(t, b, "ctx", nil, true, true, CodecBatched)
roundTrip(t, cl, ex, 5*time.Second, func() {
// A classic frame arrives on the exit's carrier.
intruder := oldClassic(t, &asyncWire{out: a.out}, CodecBatched, "ctx", false)
_ = intruder.Start()
_ = intruder.Send(testIPv4(60, 6))
time.Sleep(200 * time.Millisecond)
if m := ex.Mode(); m != "session" {
t.Errorf("exit switched to %q while its Session client was active", m)
}
})
}
func TestKDFContextsRule(t *testing.T) {
cases := []struct {
name, explicit, url string
specs []ContextSource
want string
}{
{"explicit", "x", "https://u", nil, "x"},
{"--url", "", "https://u", []ContextSource{{Type: "mailru", URL: "https://m"}}, "https://u"},
{"highest priority", "", ContextPlaceholder, []ContextSource{
{Type: "mailru", URL: "https://m", Priority: 10}, {Type: "vyandex", URL: "https://v", Priority: 100}}, "https://v"},
{"classic cups rooms as --url", "", "rooms", []ContextSource{{Type: "cupsonline", URL: "rooms"}}, ContextPlaceholder},
{"direct address", "", ContextPlaceholder, []ContextSource{{Type: "direct", URL: "1.2.3.4:9443"}}, ContextPlaceholder},
}
for _, c := range cases {
got, alts := KDFContexts(c.explicit, c.url, c.specs)
if got != c.want {
t.Errorf("%s: %q, want %q", c.name, got, c.want)
}
for _, a := range alts {
if a == got {
t.Errorf("%s: primary repeated among alternates", c.name)
}
}
}
_, alts := KDFContexts("", "rooms", []ContextSource{{Type: "cupsonline", URL: "rooms"}})
if !bytes.Contains([]byte(joinStrings(alts)), []byte("rooms")) {
t.Errorf("old classic cups context missing from alternates %q", alts)
}
}
func joinStrings(s []string) string {
var b bytes.Buffer
for _, x := range s {
b.WriteString(x)
b.WriteByte('|')
}
return b.String()
}
transport/control/cookies.go
+11 -4
View File
@@ -10,10 +10,15 @@ import "encoding/json"
type CookiesPayload struct {
// Transport names the transport the cookies belong to. Empty (older
// peers) means the highest-priority transport that carries cookies.
Transport string `json:"transport,omitempty"`
Jar map[string]string `json:"jar,omitempty"`
Domain string `json:"domain,omitempty"`
Reason string `json:"reason,omitempty"`
Transport string `json:"transport,omitempty"`
// Doc is the document URL of that transport on the sender's side, so
// the receiver can find its own carrier when the two sides name them
// differently (an app naming carriers after their type, a .conf naming
// sections freely). Optional; older peers ignore it.
Doc string `json:"doc,omitempty"`
Jar map[string]string `json:"jar,omitempty"`
Domain string `json:"domain,omitempty"`
Reason string `json:"reason,omitempty"`
}
// Encode serializes the payload to JSON.
@@ -38,6 +43,8 @@ type AuthRequiredPayload struct {
Transport string `json:"transport"`
URL string `json:"url"`
Reason string `json:"reason"`
// Doc: see CookiesPayload.Doc.
Doc string `json:"doc,omitempty"`
}
// Encode serializes the payload to JSON.
transport/encrypted.go
+360 -56
View File
@@ -11,6 +11,7 @@ import (
"fmt"
"sync"
"sync/atomic"
"time"
"golang.org/x/crypto/scrypt"
@@ -25,6 +26,18 @@ const (
var encryptedMagic = [3]byte{'O', 'F', 'X'}
// Context fallback timing. A client that has not heard a single valid
// packet within contextFirstRotate of its first send moves on to the next
// candidate context every contextRotateEvery while it keeps sending; one
// that has, but then hears nothing for
// contextUnlockAfter while sending, starts trying again (the exit may have
// been reconfigured).
const (
contextFirstRotate = 4 * time.Second
contextRotateEvery = 2 * time.Second
contextUnlockAfter = 2 * time.Minute
)
// EncryptedTransport wraps another Transport with end-to-end AES-256-GCM
// authenticated encryption, so the transport's own provider only ever
// observes ciphertext. Keys are derived from a shared secret via scrypt; the
@@ -36,15 +49,22 @@ var encryptedMagic = [3]byte{'O', 'F', 'X'}
// Every packet also carries a random nonce and is checked against a bounded
// replay window, so a captured packet cannot be replayed back at either
// peer.
//
// Context fallback: builds have derived the context differently (see
// KDFContexts), and a mismatch used to drop every packet without a word.
// SetAlternateContexts gives the transport the contexts a peer may have
// used instead. A packet that fails under the current keys is tried under
// the others; the exit then answers under whichever the client used, and a
// client that hears nothing cycles through them until the exit answers.
// This weakens nothing: the context is a public salt and every candidate
// still requires the secret.
type EncryptedTransport struct {
Transport
sendAEAD cipher.AEAD
receiveAEAD cipher.AEAD
sendDirection byte
recvDirection byte
seenMu sync.Mutex
seen map[string]struct{}
seenOrder []string
ring *keyRing
seenMu sync.Mutex
seen map[string]struct{}
seenOrder []string
// Diagnostic counters.
sendOK atomic.Uint64
@@ -56,6 +76,47 @@ type EncryptedTransport struct {
recvReplay atomic.Uint64
}
// contextKeys are the AEADs derived for one candidate context.
type contextKeys struct {
context string
send cipher.AEAD
recv cipher.AEAD
}
// keyStore holds the keys of every candidate context of one carrier,
// derived once and shared by the EncryptedTransports on it (a Session's
// and its classic fallback's).
type keyStore struct {
secret string
exit bool
side string
sendDir byte
recvDir byte
mu sync.Mutex
keys []*contextKeys // [0] is the primary context
pending []string // candidates not derived yet
deriving bool
}
// keyRing is one EncryptedTransport's view of the store: which context it
// sends under and what it has heard. Each pipeline searches on its own (a
// Session's hellos going unanswered by a classic exit must not move the
// classic pipeline off the right context); a client's pipelines tell each
// other what they learn, since they talk to the same exit.
type keyRing struct {
*keyStore
mu sync.Mutex
send int // index into keys used for sending
locked bool // the peer answered under keys[send]
heard time.Time
started time.Time // first send while nothing was heard
rotated time.Time
failures uint64 // packets no candidate could open since the last success
siblings []*keyRing
}
// NewEncryptedTransport wraps inner with a directional AES-256-GCM stream.
// Both peers must be configured with the same secret and context, and
// exactly one of them must set exitNode=true so the two sides pick opposite
@@ -64,50 +125,113 @@ func NewEncryptedTransport(inner Transport, secret, context string, exitNode boo
if inner == nil {
return nil, errors.New("inner transport is nil")
}
if len(secret) < 16 {
return nil, errors.New("encryption secret must contain at least 16 characters")
if utils.SecretChars(secret) < utils.MinSecretChars {
return nil, fmt.Errorf("encryption secret must contain at least %d characters", utils.MinSecretChars)
}
side := "CLIENT"
st := &keyStore{secret: secret, exit: exitNode, side: "CLIENT", sendDir: 0, recvDir: 1}
if exitNode {
side = "EXIT"
st.side, st.sendDir, st.recvDir = "EXIT", 1, 0
}
k, err := st.derive(context)
if err != nil {
return nil, err
}
st.keys = []*contextKeys{k}
return &EncryptedTransport{Transport: inner, ring: &keyRing{keyStore: st}, seen: make(map[string]struct{})}, nil
}
// SharingKeys returns an EncryptedTransport over inner that shares e's
// derived keys (not its replay window or its context search). Used for the
// classic fallback path that runs next to a Session on the same carrier.
func (e *EncryptedTransport) SharingKeys(inner Transport) *EncryptedTransport {
r := &keyRing{keyStore: e.ring.keyStore}
if !e.ring.exit {
// The client's pipelines reach the same exit: what one learns
// about its context holds for the other. The exit's may serve
// different clients, so each follows its own.
e.ring.mu.Lock()
e.ring.siblings = append(e.ring.siblings, r)
r.siblings = []*keyRing{e.ring}
e.ring.mu.Unlock()
}
return &EncryptedTransport{Transport: inner, ring: r, seen: make(map[string]struct{})}
}
// SetAlternateContexts sets the contexts the peer may derive its keys from
// instead of the primary one (see KDFContexts). They are derived only when
// needed.
func (e *EncryptedTransport) SetAlternateContexts(contexts []string) {
st := e.ring.keyStore
st.mu.Lock()
defer st.mu.Unlock()
have := make(map[string]bool)
for _, k := range st.keys {
have[k.context] = true
}
for _, c := range st.pending {
have[c] = true
}
for _, c := range contexts {
if c != "" && !have[c] {
have[c] = true
st.pending = append(st.pending, c)
}
}
if len(st.pending) > 0 {
utils.Debugf("[CRYPTO] side=%s %d alternate KDF context(s) on standby (derived on demand)", st.side, len(st.pending))
}
}
// Context returns the context this side currently sends under.
func (e *EncryptedTransport) Context() string {
r := e.ring
r.mu.Lock()
idx := r.send
r.mu.Unlock()
return r.key(idx).context
}
// snapshot returns the derived keys and whether candidates are pending.
func (st *keyStore) snapshot() ([]*contextKeys, bool) {
st.mu.Lock()
defer st.mu.Unlock()
return append([]*contextKeys(nil), st.keys...), len(st.pending) > 0
}
func (st *keyStore) key(i int) *contextKeys {
st.mu.Lock()
defer st.mu.Unlock()
return st.keys[i]
}
func (st *keyStore) derive(context string) (*contextKeys, error) {
salt := sha256.Sum256([]byte("OpenFlux encrypted transport v1\x00" + context))
master, err := scrypt.Key([]byte(secret), salt[:], 32768, 8, 1, 32)
master, err := scrypt.Key([]byte(st.secret), salt[:], 32768, 8, 1, 32)
if err != nil {
return nil, fmt.Errorf("derive encryption key: %w", err)
}
clientToExit := deriveDirectionalKey(master, "client-to-exit")
exitToClient := deriveDirectionalKey(master, "exit-to-client")
sendKey, receiveKey := clientToExit, exitToClient
sendDirection, receiveDirection := byte(0), byte(1)
if exitNode {
if st.exit {
sendKey, receiveKey = exitToClient, clientToExit
sendDirection, receiveDirection = 1, 0
}
sendAEAD, err := newGCM(sendKey)
send, err := newGCM(sendKey)
if err != nil {
return nil, err
}
receiveAEAD, err := newGCM(receiveKey)
recv, err := newGCM(receiveKey)
if err != nil {
return nil, err
}
// --- Diagnostic dumps ---------------------------------------------------
//
// [KEYDUMP] digest is emitted from debug level 2 (-dd): SHA-256
// prefixes of the context and of the derived keys, so two peers can
// compare derivations without revealing keys. It carries no hash of
// the secret itself: that would let anyone with the log test guesses
// without paying for scrypt.
//
// The detailed dump (secret hex, master, directional keys, send/recv
// keys) is printed only when --sensitive is set.
utils.Debugf("[KEYDUMP] digest side=%s contextSHA256=%s masterSHA256=%s c2eSHA256=%s e2cSHA256=%s",
side,
// without paying for scrypt. The detailed dump is --sensitive only.
utils.Debugf("[KEYDUMP] digest side=%s context=%q contextSHA256=%s masterSHA256=%s c2eSHA256=%s e2cSHA256=%s",
st.side, context,
utils.Sha256Short([]byte(context)),
utils.Sha256Short(master),
utils.Sha256Short(clientToExit),
@@ -115,30 +239,185 @@ func NewEncryptedTransport(inner Transport, secret, context string, exitNode boo
)
if utils.Sensitive() {
utils.Debugf("[KEYDUMP] ============================================================")
utils.Debugf("[KEYDUMP] side=%s", side)
utils.Debugf("[KEYDUMP] secretLen=%d", len(secret))
utils.Debugf("[KEYDUMP] secretSHA256=%s", utils.Sha256Hex([]byte(secret)))
utils.Debugf("[KEYDUMP] secretHex(first 32)=%s", hex.EncodeToString([]byte(secret)[:minInt(len(secret), 32)]))
utils.Debugf("[KEYDUMP] side=%s", st.side)
utils.Debugf("[KEYDUMP] secretLen=%d", len(st.secret))
utils.Debugf("[KEYDUMP] secretSHA256=%s", utils.Sha256Hex([]byte(st.secret)))
utils.Debugf("[KEYDUMP] secretHex(first 32)=%s", hex.EncodeToString([]byte(st.secret)[:minInt(len(st.secret), 32)]))
utils.Debugf("[KEYDUMP] context=%q", context)
utils.Debugf("[KEYDUMP] contextSHA256=%s", utils.Sha256Hex([]byte(context)))
utils.Debugf("[KEYDUMP] salt=%s", hex.EncodeToString(salt[:]))
utils.Debugf("[KEYDUMP] master=%s", hex.EncodeToString(master))
utils.Debugf("[KEYDUMP] client->exit=%s", hex.EncodeToString(clientToExit))
utils.Debugf("[KEYDUMP] exit->client=%s", hex.EncodeToString(exitToClient))
utils.Debugf("[KEYDUMP] sendDir=%d recvDir=%d", sendDirection, receiveDirection)
utils.Debugf("[KEYDUMP] sendDir=%d recvDir=%d", st.sendDir, st.recvDir)
utils.Debugf("[KEYDUMP] sendKey=%s", hex.EncodeToString(sendKey))
utils.Debugf("[KEYDUMP] recvKey=%s", hex.EncodeToString(receiveKey))
utils.Debugf("[KEYDUMP] ============================================================")
}
return &contextKeys{context: context, send: send, recv: recv}, nil
}
return &EncryptedTransport{
Transport: inner,
sendAEAD: sendAEAD,
receiveAEAD: receiveAEAD,
sendDirection: sendDirection,
recvDirection: receiveDirection,
seen: make(map[string]struct{}),
}, nil
// deriveMore derives the pending candidates in the background, one at a
// time (each scrypt run takes 32 MiB).
func (st *keyStore) deriveMore() {
st.mu.Lock()
if st.deriving || len(st.pending) == 0 {
st.mu.Unlock()
return
}
st.deriving = true
st.mu.Unlock()
utils.SafeGo("crypto.deriveAlternates", func() {
defer func() {
st.mu.Lock()
st.deriving = false
st.mu.Unlock()
}()
for {
st.mu.Lock()
if len(st.pending) == 0 {
st.mu.Unlock()
return
}
c := st.pending[0]
st.pending = st.pending[1:]
st.mu.Unlock()
k, err := st.derive(c)
if err != nil {
utils.Debugf("[CRYPTO] derive alternate context %q: %v", c, err)
continue
}
st.mu.Lock()
st.keys = append(st.keys, k)
n := len(st.keys)
st.mu.Unlock()
utils.Debugf("[CRYPTO] side=%s alternate KDF context #%d ready: %q", st.side, n-1, c)
}
})
}
// sendKeys picks the keys to send under. A client that has not heard the
// peer under any context moves to the next candidate every
// contextRotateEvery; an exit answers under the context the client last
// used.
func (r *keyRing) sendKeys() *contextKeys {
keys, pending := r.snapshot()
r.mu.Lock()
defer r.mu.Unlock()
now := time.Now()
if !r.exit {
if r.locked && now.Sub(r.heard) > contextUnlockAfter {
r.locked = false
r.started, r.rotated = time.Time{}, time.Time{}
utils.Debugf("[CRYPTO] side=%s peer silent for %v: will try the other KDF contexts again", r.side, contextUnlockAfter)
}
if !r.locked && (len(keys) > 1 || pending) {
if r.started.IsZero() {
r.started, r.rotated = now, now
} else if now.Sub(r.rotated) >= contextRotateEvery && now.Sub(r.started) >= contextFirstRotate {
r.rotated = now
if pending {
go r.deriveMore()
}
if len(keys) > 1 {
prev := r.send
r.send = (r.send + 1) % len(keys)
if utils.Throttled("crypto.rotate."+r.side, 10*time.Second) {
utils.Infof("[CRYPTO] no answer from the peer under KDF context %q for %v; trying %q",
keys[prev].context, contextRotateEvery, keys[r.send].context)
}
}
}
}
}
return keys[r.send]
}
// open tries every derived context, the one in use first. It reports the
// plaintext and the index of the context that opened the packet.
func (r *keyRing) open(nonce, ciphertext, header []byte) ([]byte, int, error) {
keys, pending := r.snapshot()
r.mu.Lock()
first := r.send
r.mu.Unlock()
plaintext, err := keys[first].recv.Open(nil, nonce, ciphertext, header)
if err == nil {
return plaintext, first, nil
}
for i, k := range keys {
if i == first {
continue
}
if p, e := k.recv.Open(nil, nonce, ciphertext, header); e == nil {
return p, i, nil
}
}
if pending {
r.deriveMore()
}
return nil, -1, err
}
// accepted records that the peer spoke under keys[idx], and tells a
// client's other pipeline.
func (r *keyRing) accepted(idx int) {
r.mu.Lock()
siblings := r.siblings
r.mu.Unlock()
r.adopt(idx, true)
for _, s := range siblings {
s.adopt(idx, false)
}
}
func (r *keyRing) adopt(idx int, heard bool) {
ctx := r.key(idx).context
primary := r.key(0).context
r.mu.Lock()
defer r.mu.Unlock()
if heard {
r.heard = time.Now()
r.failures = 0
}
if idx == r.send && (r.locked || r.exit) {
return
}
prev := r.key(r.send).context
r.send = idx
r.locked = true
if !heard {
return
}
switch {
case prev == ctx:
utils.Debugf("[CRYPTO] side=%s peer confirmed KDF context %q", r.side, ctx)
case idx == 0:
utils.Debugf("[CRYPTO] side=%s peer is back on the primary KDF context %q", r.side, ctx)
case utils.Throttled("crypto.adopt."+r.side, 10*time.Second):
utils.Infof("[CRYPTO] peer derives its keys from KDF context %q, not %q as this side does: switched to it (both sides should be updated to the same core)",
ctx, primary)
}
}
// failed counts a packet no candidate context could open and explains the
// likely cause now and then.
func (r *keyRing) failed() {
keys, pending := r.snapshot()
r.mu.Lock()
r.failures++
n := r.failures
heard := !r.heard.IsZero()
r.mu.Unlock()
if n < 3 || pending || !utils.Throttled("crypto.fail."+r.side, 30*time.Second) {
return
}
if heard {
utils.Infof("[CRYPTO] %d packets from the peer failed authentication after it had worked: another device with a different key, or old data replayed on the carrier", n)
return
}
utils.Infof("[CRYPTO] %d packets from the peer failed authentication under all %d KDF contexts tried (primary %q, sha256 %s): the encryption key differs from the peer's",
n, len(keys), keys[0].context, utils.Sha256Short([]byte(keys[0].context)))
}
func deriveDirectionalKey(master []byte, label string) []byte {
@@ -160,20 +439,22 @@ func newGCM(key []byte) (cipher.AEAD, error) {
}
func (e *EncryptedTransport) Send(data []byte) error {
header := []byte{encryptedMagic[0], encryptedMagic[1], encryptedMagic[2], encryptedVersion, e.sendDirection}
nonce := make([]byte, e.sendAEAD.NonceSize())
r := e.ring
keys := r.sendKeys()
header := []byte{encryptedMagic[0], encryptedMagic[1], encryptedMagic[2], encryptedVersion, r.sendDir}
nonce := make([]byte, keys.send.NonceSize())
if _, err := rand.Read(nonce); err != nil {
utils.Debugf("[CRYPTO] Send: rand nonce failed: %v", err)
e.sendErr.Add(1)
return fmt.Errorf("create packet nonce: %w", err)
}
packet := make([]byte, 0, len(header)+len(nonce)+len(data)+e.sendAEAD.Overhead())
packet := make([]byte, 0, len(header)+len(nonce)+len(data)+keys.send.Overhead())
packet = append(packet, header...)
packet = append(packet, nonce...)
packet = e.sendAEAD.Seal(packet, nonce, data, header)
packet = keys.send.Seal(packet, nonce, data, header)
utils.Debugf("[CRYPTO] Send #%d dir=%d plaintext=%d ciphertext=%d nonce=%s",
e.sendOK.Load()+1, e.sendDirection, len(data), len(packet), hex.EncodeToString(nonce))
utils.Debugf("[CRYPTO] Send #%d dir=%d plaintext=%d ciphertext=%d nonce=%s ctx=%s",
e.sendOK.Load()+1, r.sendDir, len(data), len(packet), hex.EncodeToString(nonce), utils.Sha256Short([]byte(keys.context)))
// Plaintext frames can carry control messages with cookie jars, so
// they are dumped only with --sensitive; ciphertext is what the
// carrier sees anyway.
@@ -195,13 +476,15 @@ func (e *EncryptedTransport) Send(data []byte) error {
}
func (e *EncryptedTransport) Receive(callback func([]byte)) {
r := e.ring
e.Transport.Receive(func(packet []byte) {
utils.Debugf("[CRYPTO] Recv raw %d ciphertext bytes", len(packet))
if utils.IsVerbose() {
utils.Debugf("[CRYPTO] Recv raw hexdump:\n%s", hex.Dump(packet))
}
minLen := encryptedHeader + e.receiveAEAD.NonceSize() + e.receiveAEAD.Overhead()
const nonceSize, overhead = 12, 16 // AES-GCM
minLen := encryptedHeader + nonceSize + overhead
if len(packet) < minLen {
e.recvBadLen.Add(1)
utils.Debugf("[CRYPTO] Recv DROP: too short (%d < %d)", len(packet), minLen)
@@ -211,8 +494,8 @@ func (e *EncryptedTransport) Receive(callback func([]byte)) {
if header[0] != encryptedMagic[0] || header[1] != encryptedMagic[1] ||
header[2] != encryptedMagic[2] {
e.recvBadHdr.Add(1)
utils.Debugf("[CRYPTO] Recv DROP: bad magic %x (want %x)",
header[:3], encryptedMagic[:])
utils.Debugf("[CRYPTO] Recv DROP: bad magic %x (want %x)%s",
header[:3], encryptedMagic[:], frameHint(packet))
return
}
if header[3] != encryptedVersion {
@@ -221,17 +504,17 @@ func (e *EncryptedTransport) Receive(callback func([]byte)) {
header[3], encryptedVersion)
return
}
if header[4] != e.recvDirection {
if header[4] != r.recvDir {
e.recvBadHdr.Add(1)
utils.Debugf("[CRYPTO] Recv DROP: wrong direction %d (want %d)",
header[4], e.recvDirection)
utils.Debugf("[CRYPTO] Recv DROP: wrong direction %d (want %d): a packet of this side's own role (echo, or both peers configured as %s)",
header[4], r.recvDir, map[bool]string{true: "exit", false: "client"}[r.exit])
return
}
nonceEnd := encryptedHeader + e.receiveAEAD.NonceSize()
nonceEnd := encryptedHeader + nonceSize
nonce := packet[encryptedHeader:nonceEnd]
utils.Debugf("[CRYPTO] Recv #%d dir=%d nonce=%s cipherLen=%d -> decrypting",
e.recvOK.Load()+e.recvFail.Load()+1, header[4], hex.EncodeToString(nonce), len(packet)-nonceEnd)
plaintext, err := e.receiveAEAD.Open(nil, nonce, packet[nonceEnd:], header)
plaintext, idx, err := r.open(nonce, packet[nonceEnd:], header)
if err != nil {
e.recvFail.Add(1)
utils.Debugf("[CRYPTO] Recv DECRYPT FAIL dir=%d nonce=%s err=%v (recvFail=%d recvOK=%d)",
@@ -239,6 +522,7 @@ func (e *EncryptedTransport) Receive(callback func([]byte)) {
if utils.IsVerbose() {
utils.Debugf("[CRYPTO] failed ciphertext hexdump:\n%s", hex.Dump(packet))
}
r.failed()
return
}
if !e.rememberNonce(nonce) {
@@ -247,9 +531,10 @@ func (e *EncryptedTransport) Receive(callback func([]byte)) {
hex.EncodeToString(nonce), e.recvReplay.Load())
return
}
r.accepted(idx)
e.recvOK.Add(1)
utils.Debugf("[CRYPTO] Recv DECRYPT OK #%d dir=%d plaintext=%d bytes nonce=%s",
e.recvOK.Load(), header[4], len(plaintext), hex.EncodeToString(nonce))
utils.Debugf("[CRYPTO] Recv DECRYPT OK #%d dir=%d plaintext=%d bytes nonce=%s ctx#%d",
e.recvOK.Load(), header[4], len(plaintext), hex.EncodeToString(nonce), idx)
if utils.IsVerbose() && utils.Sensitive() {
utils.Debugf("[CRYPTO] Recv plaintext hexdump:\n%s", hex.Dump(plaintext))
}
@@ -257,6 +542,25 @@ func (e *EncryptedTransport) Receive(callback func([]byte)) {
})
}
// frameHint names what a frame that is not an encrypted record looks like,
// for the drop log: most often the peer runs the other layering (classic vs
// Session) or has no encryption at all.
func frameHint(p []byte) string {
if len(p) == 0 {
return ""
}
switch p[0] {
case batchFormatVersion:
return " - a batch-v2 frame: the peer runs classic mode (codec outside encryption)"
case 0x00, CompressionMarker:
return " - a legacy per-packet frame: the peer runs classic mode with --codec=legacy"
}
if p[0]>>4 == 4 {
return " - a bare IPv4 packet: the peer runs without encryption"
}
return ""
}
func (e *EncryptedTransport) rememberNonce(nonce []byte) bool {
key := string(nonce)
e.seenMu.Lock()
transport/framedemux.go
+115
View File
@@ -0,0 +1,115 @@
package transport
import (
"sync"
"time"
"openflux/utils"
)
// frameDemux splits one carrier between the two layerings OpenFlux has
// (see PROTOCOL_NEGOTIATION.md):
//
// Session carrier <- AES-GCM record ("OFX...") <- batch-v2 <- envelopes
// classic carrier <- codec frame (0x02 / 0x00 / 0x1F) <- AES-GCM <- IPv4
//
// The first byte tells them apart, so a Session and its classic fallback
// share a carrier without either seeing the other's frames. Only the
// Session side starts and stops the carrier.
type frameDemux struct {
raw Transport
name string
once sync.Once
mu sync.RWMutex
sessionCb func([]byte)
classicCb func([]byte)
}
func newFrameDemux(raw Transport, name string) *frameDemux {
return &frameDemux{raw: raw, name: name}
}
func (d *frameDemux) attach() {
d.once.Do(func() { d.raw.Receive(d.dispatch) })
}
func (d *frameDemux) dispatch(p []byte) {
if len(p) == 0 {
return
}
d.mu.RLock()
sessionCb, classicCb := d.sessionCb, d.classicCb
d.mu.RUnlock()
if p[0] == encryptedMagic[0] {
if sessionCb != nil {
sessionCb(p)
}
return
}
if classicCb != nil {
classicCb(p)
return
}
if len(p) == 1 && p[0] == 0x00 {
return // a carrier keepalive (Volga), not a peer frame
}
if utils.Throttled("demux.classic."+d.name, 30*time.Second) {
utils.Infof("[SESSION] %q: the peer sends classic-mode frames (first byte 0x%02x)%s, but this side serves Session peers only "+
"(--negotiate, --transports, .conf transports, a wizard node): update the peer to a build on this core, give it a Session "+
"profile, or run this side classic (--transport=X with a key), which serves both",
d.name, p[0], classicFrameKind(p[0]))
}
}
func classicFrameKind(b byte) string {
switch b {
case batchFormatVersion:
return " - batch-v2"
case 0x00, CompressionMarker:
return " - legacy per-packet"
}
if b>>4 == 4 {
return " - unencrypted IPv4"
}
return ""
}
func (d *frameDemux) side(session bool) *demuxSide {
return &demuxSide{d: d, session: session}
}
// demuxSide is one layering's view of the carrier.
type demuxSide struct {
d *frameDemux
session bool
}
func (s *demuxSide) Start() error {
if s.session {
return s.d.raw.Start()
}
return nil
}
func (s *demuxSide) Stop() error {
if s.session {
return s.d.raw.Stop()
}
return nil
}
func (s *demuxSide) Send(p []byte) error { return s.d.raw.Send(p) }
func (s *demuxSide) IsConnected() bool { return s.d.raw.IsConnected() }
func (s *demuxSide) Stats() TransportStats { return s.d.raw.Stats() }
func (s *demuxSide) Receive(cb func([]byte)) {
s.d.mu.Lock()
if s.session {
s.d.sessionCb = cb
} else {
s.d.classicCb = cb
}
s.d.mu.Unlock()
s.d.attach()
}
transport/kdfcontext.go
+106
View File
@@ -0,0 +1,106 @@
package transport
import "sort"
// ContextPlaceholder is the --url default and the encryption context of a
// channel none of whose carriers has a document URL.
const ContextPlaceholder = "http://#"
// maxContextCandidates bounds how many contexts a peer tries: each one costs
// an scrypt derivation (32 MiB, tens of milliseconds) the first time it is
// needed.
const maxContextCandidates = 8
// ContextSource is one carrier as far as the encryption context cares.
type ContextSource struct {
Type string
URL string
Priority int
}
// contextURL reports whether a carrier's URL can name the channel. A
// cupsonline "URL" is the room list its exit creates at start, so the exit
// cannot know it beforehand; direct carries host:port, which differs between
// the two sides (0.0.0.0:port on the exit, the public address on the
// client); oneme has no URL.
func contextURL(s ContextSource) bool {
switch s.Type {
case "cupsonline", "direct", "oneme":
return false
}
return s.URL != "" && s.URL != ContextPlaceholder
}
// KDFContexts is the one rule every OpenFlux peer uses to pick the context
// its encryption keys are derived from (the scrypt salt); both peers must
// arrive at the same string. It replaces the copies that had drifted apart
// (CLI, Android bridge, Desktop, iOS):
//
// explicit --session-context / the context an openflux:// link carries
// --url unless it is a cupsonline room list
// transports URL of the highest-priority carrier that names the
// channel (see contextURL); equal priorities keep the first
// fallback ContextPlaceholder
//
// alternates are the contexts other or older builds derive for the same
// setup, most likely first: the placeholder, --url taken literally (older
// classic cupsonline peers used the room list), every carrier URL, and the
// transport names (third-party panels and an older fork used them when no
// URL was set). A receiver that cannot authenticate a packet under the
// primary context tries these before dropping it; see EncryptedTransport.
func KDFContexts(explicit, globalURL string, specs []ContextSource) (primary string, alternates []string) {
cups := false
for _, s := range specs {
if s.Type == "cupsonline" && s.URL != "" && s.URL == globalURL {
cups = true
}
}
if len(specs) == 1 && specs[0].Type == "cupsonline" {
cups = true
}
switch {
case explicit != "":
primary = explicit
case globalURL != "" && globalURL != ContextPlaceholder && !cups:
primary = globalURL
default:
best := -1
for i, s := range specs {
if !contextURL(s) {
continue
}
if best < 0 || s.Priority > specs[best].Priority {
best = i
}
}
if best >= 0 {
primary = specs[best].URL
} else {
primary = ContextPlaceholder
}
}
seen := map[string]bool{primary: true}
add := func(c string) {
if c == "" || seen[c] || len(alternates) >= maxContextCandidates-1 {
return
}
seen[c] = true
alternates = append(alternates, c)
}
add(ContextPlaceholder)
if globalURL != ContextPlaceholder {
add(globalURL)
}
byPriority := append([]ContextSource(nil), specs...)
sort.SliceStable(byPriority, func(i, j int) bool { return byPriority[i].Priority > byPriority[j].Priority })
for _, s := range byPriority {
if s.Type != "direct" && s.Type != "oneme" {
add(s.URL)
}
}
for _, s := range byPriority {
add(s.Type)
}
return primary, alternates
}
transport/mailru/mailru.go
+16 -1
View File
@@ -56,9 +56,15 @@ type DocSession struct {
func (s *DocSession) safeWrite(messageType int, data []byte) error {
s.writeMu.Lock()
defer s.writeMu.Unlock()
// A write into a half-open connection (NAT dropped it, the network
// changed) would otherwise block until the kernel gives up, minutes.
_ = s.Conn.SetWriteDeadline(time.Now().Add(docWriteTimeout))
return s.Conn.WriteMessage(messageType, data)
}
// docWriteTimeout bounds one WebSocket write to the document.
const docWriteTimeout = 20 * time.Second
type MailruDocsTransport struct {
*transport.BaseTransport
@@ -163,6 +169,9 @@ func (t *MailruDocsTransport) connectToDoc(attempt int) {
info, err := t.fetchDocInfo(t.weblink)
if err != nil {
utils.Debugf("[M-DOCS] fetchDocInfo failed: %v", err)
if utils.Throttled("m-docs.fetch", time.Minute) {
utils.Infof("[M-DOCS] cannot open the document: %v; retrying", err)
}
t.scheduleReconnect(attempt)
return
}
@@ -267,6 +276,9 @@ func (t *MailruDocsTransport) connectToDoc(attempt int) {
_, message, err := conn.ReadMessage()
if err != nil {
utils.Debugf("[M-DOCS] Read error: %v", err)
if utils.Throttled("m-docs.drop", time.Minute) {
utils.Infof("[M-DOCS] connection to the document dropped: %v; reconnecting", err)
}
t.SetConnected(false)
conn.Close()
@@ -343,8 +355,11 @@ func (t *MailruDocsTransport) keepAliveLoop() {
if session != nil && session.Conn != nil {
if err := session.safeWrite(websocket.TextMessage, []byte(keepAliveMsg)); err != nil {
utils.Debugf("[M-DOCS] Keep-alive failed: %v", err)
utils.Debugf("[M-DOCS] Keep-alive failed, closing the connection to reconnect: %v", err)
t.SetConnected(false)
// Close it so the reader, which may sit in ReadMessage on
// a half-open socket forever, errors out and reconnects.
_ = session.Conn.Close()
}
}
}
transport/manager/manager.go
+74 -13
View File
@@ -10,6 +10,8 @@ package manager
import (
"errors"
"fmt"
"strconv"
"strings"
"sync"
"time"
@@ -34,6 +36,7 @@ type CookieProvider interface {
type Entry struct {
Name string
Type string
URL string // document URL, "" when the carrier has none
Priority int
Raw transport.Transport
Provider CookieProvider // nil if the transport does not carry cookies
@@ -120,6 +123,16 @@ func (m *Manager) Add(name, typ string, raw transport.Transport, priority int, p
return nil
}
// SetURL records the document URL of an attached transport, which cookie
// messages carry so the peer can match carriers it names differently.
func (m *Manager) SetURL(name, url string) {
m.mu.Lock()
if e, ok := m.entries[name]; ok {
e.URL = url
}
m.mu.Unlock()
}
// Remove detaches a transport from the Session and stops it.
func (m *Manager) Remove(name string) error {
m.mu.Lock()
@@ -299,20 +312,64 @@ func shareableCookies(jar map[string]string) map[string]string {
return out
}
// cookieTransport resolves the transport a cookie message refers to: the
// named one, or for peers that predate named messages, the
// highest-priority transport that carries cookies.
func (m *Manager) cookieTransport(name string) string {
// cookieTransport resolves the transport a cookie message refers to. The
// two sides do not always name carriers alike (the apps name them after the
// type, "mailru", "mailru-2"; a .conf or a panel names sections freely), so
// after the name itself it tries the document URL the message carries,
// then the type the name starts with when this side has one carrier of it.
// Peers that predate named messages mean the highest-priority transport
// that carries cookies.
func (m *Manager) cookieTransport(name, doc string) string {
m.mu.RLock()
defer m.mu.RUnlock()
if name != "" {
if name == "" {
for _, n := range m.order {
if m.entries[n].Provider != nil {
return n
}
}
return ""
}
if _, ok := m.entries[name]; ok {
return name
}
for _, n := range m.order {
if m.entries[n].Provider != nil {
return n
if doc != "" {
for _, n := range m.order {
if m.entries[n].URL == doc {
utils.Debugf("[MANAGER] peer's carrier %q is this side's %q (same document)", name, n)
return n
}
}
}
typ := name
if i := strings.LastIndex(name, "-"); i > 0 {
if _, err := strconv.Atoi(name[i+1:]); err == nil {
typ = name[:i]
}
}
var match []string
for _, n := range m.order {
if m.entries[n].Type == typ {
match = append(match, n)
}
}
if len(match) == 1 {
utils.Debugf("[MANAGER] peer's carrier %q is this side's %q (only %s)", name, match[0], typ)
return match[0]
}
if utils.Throttled("manager.name."+name, time.Minute) {
utils.Infof("[MANAGER] the peer refers to carrier %q, which this side cannot match (carriers: %v): the two configs name carriers differently", name, m.order)
}
return ""
}
// entryURL is the document URL of an attached transport.
func (m *Manager) entryURL(name string) string {
m.mu.RLock()
defer m.mu.RUnlock()
if e, ok := m.entries[name]; ok {
return e.URL
}
return ""
}
@@ -324,7 +381,7 @@ func (m *Manager) handleCookies(sub control.Subtype, payload []byte) {
utils.Debugf("[MANAGER] bad cookies payload: %v", err)
return
}
name := m.cookieTransport(cp.Transport)
name := m.cookieTransport(cp.Transport, cp.Doc)
if name == "" {
return
}
@@ -339,7 +396,7 @@ func (m *Manager) handleCookies(sub control.Subtype, payload []byte) {
return
}
jar = shareableCookies(jar)
body, _ := (&control.CookiesPayload{Transport: name, Jar: jar, Reason: "requested"}).Encode()
body, _ := (&control.CookiesPayload{Transport: name, Doc: m.entryURL(name), Jar: jar, Reason: "requested"}).Encode()
_ = m.SendControl(control.SubtypeCookiesResponse, body)
case control.SubtypeCookiesResponse, control.SubtypeCookiesOffer:
if len(cp.Jar) == 0 {
@@ -394,8 +451,12 @@ func (m *Manager) DispatchControl(sub control.Subtype, payload []byte) {
return
}
// The exit's side of that transport is stuck: stop routing through
// it here as well, the cookies that unstick it included.
m.session.MarkStalled(req.Transport)
// it here as well, the cookies that unstick it included. The report
// keeps the exit's name: the cookies go back under it.
if local := m.cookieTransport(req.Transport, req.Doc); local != "" {
m.session.MarkStalled(local)
}
utils.Infof("[MANAGER] the exit's carrier %q needs a check in a browser (%s): %s", req.Transport, req.Reason, req.URL)
m.mu.RLock()
cb := m.remoteAuth
m.mu.RUnlock()
@@ -552,7 +613,7 @@ func (m *Manager) forwardAuth(name, url, reason string) {
m.authSent[name] = now
m.mu.Unlock()
body, _ := (&control.AuthRequiredPayload{Transport: name, URL: url, Reason: reason}).Encode()
body, _ := (&control.AuthRequiredPayload{Transport: name, URL: url, Reason: reason, Doc: m.entryURL(name)}).Encode()
if err := m.SendControl(control.SubtypeAuthRequired, body); err != nil {
// No client yet: let the transport's next report try again.
utils.Debugf("[MANAGER] forward AuthRequired (%s): %v", name, err)
transport/session.go
+419 -25
View File
@@ -64,8 +64,43 @@ type Session struct {
cntCtrlRecv atomic.Uint64
cntDecodeErr atomic.Uint64
cntUnknownKind atomic.Uint64
// Classic fallback, see SetClassic.
classic ClassicMode
classicCodec string
classicLink *transportLink // exit: the carrier a classic client was last heard on
classicSeen bool // client: the exit answered in classic mode
classicSince time.Time // client: when data started going out classic
alternates []string // KDF contexts the peer may derive instead, see KDFContexts
cntClassicSent atomic.Uint64
cntClassicRecv atomic.Uint64
cntClassicDrop atomic.Uint64
}
// ClassicMode is how a Session treats peers of the classic (pre-Session)
// layering.
type ClassicMode int
const (
// ClassicOff: Session peers only (--negotiate, and every exit that was
// configured as a Session). Classic frames are dropped with a log line.
ClassicOff ClassicMode = iota
// ClassicFallback (client, one carrier): until the exit answers the
// Session handshake, IPv4 goes out in the classic layering, so an exit
// that predates Session, or runs classic, still works; the client keeps
// offering the handshake and switches to the Session once answered.
ClassicFallback
// ClassicAccept (exit configured classic): the exit also serves
// classic clients, as long as no Session client is active. Session
// clients get a Session, so an updated client never runs classic.
ClassicAccept
)
// classicWait is how long a client with classic fallback waits for the
// Session handshake before letting data go out classic.
const classicWait = 3 * time.Second
type candidatePeer struct {
sender [32]byte
local [32]byte
@@ -80,15 +115,95 @@ const (
type transportLink struct {
name string
raw Transport
demux *frameDemux
encrypted *EncryptedTransport
batched *BatchedTransport
priority int
// The classic pipeline on the same carrier, nil unless SetClassic:
// classicEnc(classicCodec(classic side of demux)).
classicEnc *EncryptedTransport
classicCodec *CodecTransport
classicHeard time.Time
started bool
lastHeard time.Time
dead bool
}
// SetClassic lets this Session also speak the classic layering: a client
// falls back to it while the exit does not answer the handshake
// (ClassicFallback), an exit serves classic clients (ClassicAccept). codec is
// the preferred classic framing (CodecBatched or CodecLegacy); the other one
// is accepted too and tried when the peer stays silent. Call before
// AddTransport.
func (s *Session) SetClassic(codec string) {
s.mu.Lock()
defer s.mu.Unlock()
if s.exit {
s.classic = ClassicAccept
} else {
s.classic = ClassicFallback
}
s.classicCodec = codec
utils.Debugf("[SESSION] classic mode: %s (preferred codec %s)", s.classic, codecName(codec != CodecLegacy))
}
func (m ClassicMode) String() string {
switch m {
case ClassicFallback:
return "fallback"
case ClassicAccept:
return "accept"
}
return "off"
}
// SetAlternateContexts gives every carrier, present and future, the KDF
// contexts the peer may derive its keys from instead of this side's (see
// KDFContexts and EncryptedTransport).
func (s *Session) SetAlternateContexts(contexts []string) {
s.mu.Lock()
s.alternates = append(s.alternates, contexts...)
links := make([]*transportLink, 0, len(s.links))
for _, l := range s.links {
links = append(links, l)
}
s.mu.Unlock()
for _, l := range links {
l.encrypted.SetAlternateContexts(contexts)
}
}
// newLink builds a carrier's pipelines: the Session's
// batched(encrypted(session side)) and, with classic mode on, the classic
// encrypted(codec(classic side)) sharing its keys.
func (s *Session) newLink(name string, raw Transport, secret, context string, priority int) (*transportLink, error) {
d := newFrameDemux(raw, name)
enc, err := NewEncryptedTransport(d.side(true), secret, context, s.exit)
if err != nil {
return nil, fmt.Errorf("session: wrap %q: %w", name, err)
}
s.mu.Lock()
alternates := append([]string(nil), s.alternates...)
classic, codec := s.classic, s.classicCodec
s.mu.Unlock()
enc.SetAlternateContexts(alternates)
link := &transportLink{
name: name,
raw: raw,
demux: d,
encrypted: enc,
batched: NewBatchedTransport(enc),
priority: priority,
}
if classic != ClassicOff {
link.classicCodec = NewCodecTransport(d.side(false), codec, !s.exit)
link.classicEnc = enc.SharingKeys(link.classicCodec)
}
return link, nil
}
func NewSession(p PeerParameters, exit bool) (*Session, error) {
if !validParameters(p) {
return nil, errors.New("session requires IPv4/TCP and a packet limit from 1280 to 65000")
@@ -146,18 +261,9 @@ func (s *Session) AddTransport(name string, raw Transport, secret, context strin
}
s.mu.Unlock()
enc, err := NewEncryptedTransport(raw, secret, context, s.exit)
link, err := s.newLink(name, raw, secret, context, priority)
if err != nil {
return fmt.Errorf("session: wrap %q: %w", name, err)
}
bat := NewBatchedTransport(enc)
link := &transportLink{
name: name,
raw: raw,
encrypted: enc,
batched: bat,
priority: priority,
return err
}
s.mu.Lock()
@@ -220,18 +326,28 @@ func (s *Session) Start() error {
timeout := s.handshakeTimeout
local := s.local
order := append([]string(nil), s.order...)
if s.classic == ClassicFallback && len(links) != 1 {
// A classic exit has exactly one carrier; several can only mean a
// Session exit.
utils.Debugf("[SESSION] classic fallback off: %d carriers (a classic exit has one)", len(links))
s.classic = ClassicOff
for _, l := range links {
l.classicEnc, l.classicCodec = nil, nil
}
}
classic := s.classic
s.mu.Unlock()
role := "client"
if exit {
role = "exit"
}
utils.Debugf("[SESSION] Start role=%s timeout=%v local=%s transports=%v",
role, timeout, shortID(local), order)
utils.Debugf("[SESSION] Start role=%s timeout=%v local=%s transports=%v classic=%s",
role, timeout, shortID(local), order, classic)
for _, link := range links {
if err := s.startLink(link); err != nil {
utils.Debugf("[SESSION] transport %q start: %v; retrying in background", link.name, err)
utils.Infof("[SESSION] carrier %q failed to start: %v; retrying in the background", link.name, err)
s.superviseLink(link)
} else {
utils.Debugf("[SESSION] transport %q started (priority=%d)", link.name, link.priority)
@@ -247,6 +363,26 @@ func (s *Session) Start() error {
s.wg.Add(1)
go s.helloLoop()
if classic == ClassicFallback {
wait := min(classicWait, timeout)
if err := s.waitReady(wait); err == nil {
s.logHandshake()
return nil
}
s.mu.Lock()
stopped := s.stopped
if !stopped && !s.ready {
s.classicSince = time.Now()
}
s.mu.Unlock()
if stopped {
return errors.New("session: stopped")
}
utils.Infof("[SESSION] no Session handshake from the exit within %v on %q: sending in classic mode meanwhile; "+
"switching to the Session as soon as the exit answers", wait, order[0])
return nil
}
utils.Debugf("[SESSION] client: waiting for handshake (timeout=%v)", timeout)
if err := s.waitReady(timeout); err != nil {
utils.Debugf("[SESSION] handshake FAILED after %v: %v", timeout, err)
@@ -255,6 +391,11 @@ func (s *Session) Start() error {
return fmt.Errorf("session: handshake failed: %w", err)
}
s.logHandshake()
return nil
}
func (s *Session) logHandshake() {
s.mu.Lock()
peer := s.peer
remote := s.remote
@@ -262,7 +403,6 @@ func (s *Session) Start() error {
utils.Debugf("[SESSION] handshake OK: peer=%s caps=0x%x maxPacket=%d",
shortID(peer), remote.Capabilities, remote.MaxPacketSize)
s.dumpDiagnostics("handshake-success")
return nil
}
func (s *Session) dumpDiagnostics(why string) {
@@ -279,11 +419,51 @@ func (s *Session) dumpDiagnostics(why string) {
name, l.started, l.dead, time.Since(l.lastHeard).Round(time.Millisecond), l.raw.IsConnected())
if l.encrypted != nil {
so, se, ro, rf, rr, bh, bl := l.encrypted.CryptoStats()
utils.Debugf("[SESSION-DIAG] crypto link=%q sendOK=%d sendErr=%d recvOK=%d recvFail=%d recvReplay=%d badHdr=%d badLen=%d",
name, so, se, ro, rf, rr, bh, bl)
utils.Debugf("[SESSION-DIAG] crypto link=%q sendOK=%d sendErr=%d recvOK=%d recvFail=%d recvReplay=%d badHdr=%d badLen=%d ctx=%q",
name, so, se, ro, rf, rr, bh, bl, l.encrypted.Context())
}
if l.classicCodec != nil {
utils.Debugf("[SESSION-DIAG] classic link=%q codec=%s heard=%v", name, l.classicCodec.Current(), !l.classicHeard.IsZero())
}
}
utils.Debugf("[SESSION-DIAG] classic mode=%s sent=%d recv=%d drop=%d", s.classic, s.cntClassicSent.Load(), s.cntClassicRecv.Load(), s.cntClassicDrop.Load())
s.mu.Unlock()
if why != "handshake-success" {
utils.Infof("[SESSION] handshake not complete (%s): %s", why, s.diagnose())
}
}
// diagnose names the likeliest reason the exit has not answered the
// handshake, from what did arrive on the carriers.
func (s *Session) diagnose() string {
s.mu.Lock()
var recv, ok, fail uint64
connected := false
for _, l := range s.links {
recv += l.raw.Stats().PacketsRecv
connected = connected || l.raw.IsConnected()
if l.encrypted != nil {
_, _, ro, rf, _, _, _ := l.encrypted.CryptoStats()
ok += ro
fail += rf
}
}
classicHeard := s.classicSeen
s.mu.Unlock()
hellos := s.cntHelloRecv.Load()
switch {
case !connected:
return "no carrier is connected on this side (document unreachable, captcha, or network)"
case classicHeard:
return "the exit answers in classic mode only (an old or classic exit); traffic flows classic"
case recv == 0:
return "nothing arrived from the exit on any carrier: it is not running, uses another document/room, or is on another carrier"
case ok == 0 && fail > 0:
return "packets arrived but none decrypted under any KDF context: the encryption key differs from the exit's"
case ok > 0 && hellos == 0:
return "packets decrypted but none was a Session hello: the exit runs classic mode"
}
return fmt.Sprintf("%d frames arrived, %d decrypted, %d hellos; see [SESSION-DIAG] at -dd", recv, ok, hellos)
}
func (s *Session) startLink(link *transportLink) error {
@@ -291,6 +471,13 @@ func (s *Session) startLink(link *transportLink) error {
return err
}
link.batched.Receive(func(p []byte) { s.receive(link, p) })
if link.classicCodec != nil {
if err := link.classicCodec.Start(); err != nil {
utils.Debugf("[SESSION] %q classic pipeline: %v", link.name, err)
} else {
link.classicEnc.Receive(func(p []byte) { s.receiveClassic(link, p) })
}
}
s.mu.Lock()
stopped := s.stopped
@@ -300,8 +487,7 @@ func (s *Session) startLink(link *transportLink) error {
}
s.mu.Unlock()
if stopped {
_ = link.batched.Stop()
_ = link.raw.Stop()
link.stop()
return errors.New("session stopped")
}
// A carrier that comes up in an established session (e.g. once a check
@@ -334,10 +520,13 @@ func (s *Session) superviseLink(link *transportLink) {
attempt++
err := s.startLink(link)
if err == nil {
utils.Debugf("[SESSION] transport %q up after %d retries", link.name, attempt)
utils.Infof("[SESSION] carrier %q up after %d retries", link.name, attempt)
return
}
utils.Debugf("[SESSION] transport %q start retry #%d: %v", link.name, attempt, err)
if utils.Throttled("session.restart."+link.name, 2*time.Minute) {
utils.Infof("[SESSION] carrier %q still not up after %d retries: %v", link.name, attempt, err)
}
if delay *= 2; delay > s.restartMax {
delay = s.restartMax
}
@@ -345,22 +534,42 @@ func (s *Session) superviseLink(link *transportLink) {
}()
}
// helloBackoff is when the hello pace drops from helloInterval to
// helloSlow; an exit proven classic gets one every helloClassic.
const (
helloBackoff = 20 * time.Second
helloSlow = 2 * time.Second
helloClassic = 10 * time.Second
)
func (s *Session) helloLoop() {
defer s.wg.Done()
tick := time.NewTicker(s.helloInterval)
defer tick.Stop()
attempt := 0
started := time.Now()
var lastSent time.Time
for {
s.mu.Lock()
ready := s.ready
pace := s.helloInterval
if time.Since(started) > helloBackoff {
pace = max(pace, helloSlow)
}
if s.classicSeen {
pace = max(pace, helloClassic)
}
var names []string
if !ready {
if !ready && time.Since(lastSent) >= pace {
for _, name := range s.order {
if s.links[name].started {
names = append(names, name)
}
}
}
if ready {
started = time.Now()
}
local := s.local
peer := s.peer
s.mu.Unlock()
@@ -371,11 +580,18 @@ func (s *Session) helloLoop() {
attempt, names, shortID(local), shortID(peer))
}
}
if len(names) > 0 {
lastSent = time.Now()
}
for _, name := range names {
if err := s.helloVia(name); err != nil {
utils.Debugf("[SESSION] hello via %q: %v", name, err)
}
}
if attempt == 80 && !ready {
attempt++
s.dumpDiagnostics("handshake-slow")
}
select {
case <-s.done:
return
@@ -397,7 +613,7 @@ func (s *Session) keepaliveLoop() {
s.mu.Lock()
if !s.exit && s.ready && s.peerKeepalive && s.peerSilentLocked() {
s.resetLocked()
utils.Debugf("[SESSION] peer silent on every transport; handshaking again")
utils.Infof("[SESSION] exit silent on every carrier for %v: handshaking again", s.linkTimeout)
}
var quiet []*transportLink
if s.ready {
@@ -446,7 +662,7 @@ func (s *Session) waitReady(timeout time.Duration) error {
tick := time.NewTicker(20 * time.Millisecond)
defer tick.Stop()
lastLog := time.Now()
for !s.IsConnected() {
for !s.handshakeDone() {
select {
case <-s.done:
return errors.New("session stopped")
@@ -480,24 +696,117 @@ func (s *Session) Stop() error {
}
s.mu.Unlock()
for _, l := range links {
_ = l.batched.Stop()
_ = l.raw.Stop()
l.stop()
}
})
s.wg.Wait()
return nil
}
// stop takes down the carrier once: the batched wrapper stops it through
// the encryption layer and the demux, the classic pipeline only its own
// queue.
func (l *transportLink) stop() {
if l.classicCodec != nil {
_ = l.classicCodec.Stop()
}
_ = l.batched.Stop()
}
func (s *Session) IsConnected() bool {
s.mu.Lock()
ready := s.ready && !s.stopped
classic := s.classicConnectedLocked()
s.mu.Unlock()
if classic {
return true
}
if !ready {
return false
}
return s.anyLive()
}
// handshakeDone reports whether the Session handshake completed and a
// carrier reaches the peer (IsConnected also counts classic mode).
func (s *Session) handshakeDone() bool {
s.mu.Lock()
ready := s.ready && !s.stopped
s.mu.Unlock()
return ready && s.anyLive()
}
// classicConnectedLocked: a classic-fallback client counts as connected
// while its carrier is (what classic mode always reported); an exit while
// a classic client was heard lately. Caller holds s.mu.
func (s *Session) classicConnectedLocked() bool {
if s.stopped || s.ready {
return false
}
switch s.classic {
case ClassicFallback:
l := s.classicTargetLocked()
return l != nil && s.connectedLocked(l)
case ClassicAccept:
l := s.classicLink
return l != nil && time.Since(l.classicHeard) < s.linkTimeout
}
return false
}
// classicTargetLocked is the carrier classic IPv4 goes out on, or nil.
// Caller holds s.mu.
func (s *Session) classicTargetLocked() *transportLink {
switch s.classic {
case ClassicFallback:
if len(s.order) == 1 {
if l := s.links[s.order[0]]; l.started && l.classicEnc != nil {
return l
}
}
case ClassicAccept:
return s.classicLink
}
return nil
}
// sessionActiveLocked reports whether the Session client was heard
// recently enough that a classic client must not take the exit's replies
// from it: a live client sends a keepalive at least every
// keepaliveInterval, so half that again covers one lost. A client that
// went away yields to a classic one after that, not after linkTimeout.
// Caller holds s.mu.
func (s *Session) sessionActiveLocked() bool {
if !s.ready {
return false
}
window := s.keepaliveInterval * 3 / 2
for _, l := range s.links {
if s.connectedLocked(l) && time.Since(l.lastHeard) < window {
return true
}
}
return false
}
// Mode names what IPv4 currently goes out as: "session", "classic" or ""
// (nothing yet).
func (s *Session) Mode() string {
s.mu.Lock()
defer s.mu.Unlock()
switch {
case s.stopped:
return ""
case s.exit && s.classicLink != nil && !s.sessionActiveLocked():
return "classic"
case s.ready:
return "session"
case s.classicTargetLocked() != nil:
return "classic"
}
return ""
}
func (s *Session) anyLive() bool {
s.mu.Lock()
defer s.mu.Unlock()
@@ -520,6 +829,11 @@ func (s *Session) heardLocked(l *transportLink) bool {
func (s *Session) ActiveTransport() string {
s.mu.Lock()
defer s.mu.Unlock()
if s.classicConnectedLocked() {
if l := s.classicTargetLocked(); l != nil {
return l.name
}
}
if !s.ready || s.stopped {
return ""
}
@@ -534,6 +848,11 @@ func (s *Session) ActiveTransport() string {
func (s *Session) LiveTransports() []string {
s.mu.Lock()
defer s.mu.Unlock()
if s.classicConnectedLocked() {
if l := s.classicTargetLocked(); l != nil {
return []string{l.name}
}
}
if !s.ready || s.stopped {
return nil
}
@@ -617,6 +936,14 @@ func (s *Session) helloVia(name string) error {
func (s *Session) Send(p []byte) error {
s.mu.Lock()
if !s.stopped && s.classic != ClassicOff {
// A client whose exit has not answered the handshake, or an exit
// whose current client is a classic one: classic layering.
if l := s.classicTargetLocked(); l != nil && (!s.ready || (s.exit && !s.sessionActiveLocked())) {
s.mu.Unlock()
return s.sendClassic(l, p)
}
}
if !s.ready || s.stopped {
s.mu.Unlock()
return ErrNegotiationPending
@@ -772,6 +1099,67 @@ func permittedPacket(p []byte, limits PeerParameters) error {
return nil
}
// ---- classic layering ----
// sendClassic sends one IPv4 packet in the classic layering on link.
func (s *Session) sendClassic(link *transportLink, p []byte) error {
if len(p) < 20 || p[0]>>4 != 4 {
return errors.New("session: classic mode carries IPv4 packets only")
}
n := s.cntClassicSent.Add(1)
if n == 1 || n%500 == 0 {
utils.Debugf("[SESSION] classic send #%d via %q size=%d codec=%s ctx=%s",
n, link.name, len(p), link.classicCodec.Current(), utils.Sha256Short([]byte(link.classicEnc.Context())))
}
return link.classicEnc.Send(p)
}
// receiveClassic takes one decrypted IPv4 packet of the classic layering.
func (s *Session) receiveClassic(link *transportLink, p []byte) {
if len(p) < 20 || p[0]>>4 != 4 {
s.cntClassicDrop.Add(1)
hint := ""
if len(p) > 0 && p[0] == 0xFF {
hint = " (a control frame of the iOS fork's own protocol, which this core does not speak; update the app to one built on this core)"
}
utils.Debugf("[SESSION] classic frame from %q is not IPv4 (%d bytes)%s", link.name, len(p), hint)
return
}
now := time.Now()
s.mu.Lock()
if s.stopped {
s.mu.Unlock()
return
}
if s.exit {
if s.sessionActiveLocked() {
s.cntClassicDrop.Add(1)
s.mu.Unlock()
if utils.Throttled("session.classic.busy", 30*time.Second) {
utils.Infof("[SESSION] classic packets on %q ignored: a Session client is active (one client per exit; another device uses the same key?)", link.name)
}
return
}
if s.classicLink != link {
utils.Infof("[SESSION] classic client on %q (it does not speak the Session handshake): serving it in classic mode; update the client to get a Session", link.name)
}
s.classicLink = link
} else if !s.classicSeen {
s.classicSeen = true
utils.Infof("[SESSION] the exit answers in classic mode on %q (it predates Session or runs classic): staying classic, still offering the handshake", link.name)
}
link.classicHeard = now
cb := s.dataCallback
s.mu.Unlock()
n := s.cntClassicRecv.Add(1)
if n == 1 || n%500 == 0 {
utils.Debugf("[SESSION] classic recv #%d from %q size=%d", n, link.name, len(p))
}
if cb != nil {
cb(append([]byte(nil), p...))
}
}
// ---- receive ----
func (s *Session) receive(link *transportLink, p []byte) {
@@ -892,6 +1280,9 @@ func (s *Session) receiveHello(link *transportLink, env *control.Envelope) {
s.mu.Unlock()
s.cntHelloAccept.Add(1)
if echo && !wasReady && !s.exit && (s.classicSeen || !s.classicSince.IsZero()) {
utils.Infof("[SESSION] the exit answered the Session handshake on %q: switching from classic to the Session", link.name)
}
if echo {
utils.Debugf("[SESSION] hello #%d from %q ACCEPT: peer=%s -> ready (accept=%d)",
n, link.name, shortID(peer), s.cntHelloAccept.Load())
@@ -930,6 +1321,9 @@ func (s *Session) offerReplacementLocked(link *transportLink, sender [32]byte, p
names := append([]string(nil), s.order...)
s.mu.Unlock()
utils.Debugf("[SESSION] peer REPLACED by %s after fresh challenge echo", shortID(sender))
if s.exit {
utils.Infof("[SESSION] a new client took over the session on %q: the previous one restarted, or two devices use the same key (the exit serves one client at a time)", link.name)
}
for _, name := range names {
_ = s.helloVia(name)
}
transport/session_add_after_start.go
+10 -15
View File
@@ -36,28 +36,23 @@ func (s *Session) AddTransportPostStart(name string, raw Transport, secret, cont
}
s.mu.Unlock()
enc, err := NewEncryptedTransport(raw, secret, context, s.exit)
link, err := s.newLink(name, raw, secret, context, priority)
if err != nil {
return fmt.Errorf("session: wrap %q: %w", name, err)
return err
}
bat := NewBatchedTransport(enc)
// Post-start carriers are Session-only: classic fallback is for a
// single-carrier client, and an exit adds these at a Session client's
// request.
link.classicEnc, link.classicCodec = nil, nil
// bat.Start starts raw through the encryption layer.
if err := bat.Start(); err != nil {
// batched.Start starts raw through the encryption layer and the demux.
if err := link.batched.Start(); err != nil {
return fmt.Errorf("session: transport %q start: %w", name, err)
}
link := &transportLink{
name: name,
raw: raw,
encrypted: enc,
batched: bat,
priority: priority,
started: true,
}
link.started = true
// Same receive path as the bootstrap transports: everything that
// arrives on this link goes through Session.receive.
bat.Receive(func(p []byte) { s.receive(link, p) })
link.batched.Receive(func(p []byte) { s.receive(link, p) })
s.mu.Lock()
s.links[name] = link
transport/yandex/boards.go
+21 -23
View File
@@ -31,8 +31,10 @@ const (
"(KHTML, like Gecko) Chrome/153.0.0.0 Mobile Safari/537.36"
boardsSocketHostDefault = "socket33.boards.yandex.ru"
// Engine.io heartbeat. Сервер шлёт pingInterval=25000, pingTimeout=30000.
// Мы пингуем сами каждые 20с, чтобы NAT не рвал idle-соединение.
// Heartbeat. Сервер шлёт engine.io ping "2" (pingInterval=25000,
// pingTimeout=30000), мы отвечаем "3". Сами шлём только heartbeat в
// namespace dashboard каждые 20с, чтобы NAT не рвал idle-соединение;
// клиентский "2" сервер EIO=4 считает ошибкой и закрывает сокет.
boardsPingInterval = 20 * time.Second
// Дедлайн чтения в основном цикле. С запасом над boardsPingInterval.
@@ -458,10 +460,20 @@ func (t *BoardsTransport) connectLoop(info boardsInfo) {
return
default:
}
if err := t.connectAndServe(info); err != nil {
utils.Debugf("[BOARDS] ws error: %v", err)
began := time.Now()
err := t.connectAndServe(info)
lasted := time.Since(began)
if err != nil {
utils.Debugf("[BOARDS] ws error after %v: %v", lasted.Round(time.Second), err)
if utils.Throttled("boards.drop", time.Minute) {
utils.Infof("[BOARDS] connection to the board dropped after %v: %v; reconnecting", lasted.Round(time.Second), err)
}
}
t.SetConnected(false)
if lasted > time.Minute {
// A session that held is not part of a failure streak.
attempt = 0
}
select {
case <-t.done:
return
@@ -547,9 +559,13 @@ func (t *BoardsTransport) connectAndServe(info boardsInfo) error {
t.SetConnected(true)
utils.SafeGo("boards.writer", func() { t.writerLoop(sess) })
// No client-side engine.io ping: in EIO=4 the server pings ("2") and the
// client answers ("3", see handleMessage). A client "2" is an invalid
// heartbeat direction to an engine.io v4 server, which then closes the
// socket, so the old pingLoop dropped the board every
// boardsPingInterval. The dashboard heartbeat below keeps it busy.
kaStop := make(chan struct{})
utils.SafeGo("boards.keepalive", func() { t.keepAliveLoop(sess, kaStop) })
utils.SafeGo("boards.ping", func() { t.pingLoop(sess, kaStop) })
defer close(kaStop)
for {
@@ -789,24 +805,6 @@ func (t *BoardsTransport) keepAliveLoop(sess *boardsSession, stop chan struct{})
}
}
func (t *BoardsTransport) pingLoop(sess *boardsSession, stop chan struct{}) {
tick := time.NewTicker(boardsPingInterval)
defer tick.Stop()
for {
select {
case <-stop:
return
case <-t.done:
return
case <-tick.C:
if err := sess.writeRaw("2"); err != nil {
utils.Debugf("[BOARDS] engine.io ping: %v", err)
return
}
}
}
}
// ---- приём ----
func (t *BoardsTransport) handleMessage(sess *boardsSession, raw []byte) {
transport/yandex/vyandex.go
+36 -4
View File
@@ -52,6 +52,11 @@ type VolgaConfig struct {
// MaxSessionAge rotates the WebSocket (and its authorization) even
// while it looks healthy; 0 disables rotation.
MaxSessionAge time.Duration
// WebSocket buffers. Large ones help a server; a phone's VPN process
// (iOS caps it at 50 MB) cannot afford them.
WSReadBufferSize int
WSWriteBufferSize int
}
func DefaultVolgaConfig() VolgaConfig {
@@ -79,16 +84,37 @@ func DefaultVolgaConfig() VolgaConfig {
WSReadTimeout: 60 * time.Second,
KeepAliveInterval: 10 * time.Second,
MaxSessionAge: 30 * time.Minute,
WSReadBufferSize: 4 << 20,
WSWriteBufferSize: 4 << 20,
}
}
// SlimVolgaConfig is the memory-constrained profile for phones, above all
// the iOS Network Extension (50 MB for the whole process): a small relay
// worker pool and queue, smaller batches and WebSocket buffers. The wire
// format is the same, so it talks to a node on the default profile.
func SlimVolgaConfig() VolgaConfig {
c := DefaultVolgaConfig()
c.MaxIdleConnsPerHost = 8
c.MaxIdleConns = 16
c.WorkerCount = 4
c.QueueSize = 4096
c.BatchMaxBytes = 256 * 1024
c.WSReadBufferSize = 128 << 10
c.WSWriteBufferSize = 128 << 10
return c
}
const volgaUserAgent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:153.0) Gecko/20100101 Firefox/153.0"
var reClientConfig = regexp.MustCompile(`<script[^>]*id="client-config"[^>]*>(.*?)</script>`)
var (
b64BufPool = sync.Pool{
New: func() interface{} { return make([]byte, 0, 16*1024*1024) },
// base64Encode grows a buffer for a bigger batch; a 16 MiB default
// kept that much per pooled buffer alive for every small packet.
New: func() interface{} { return make([]byte, 0, 256*1024) },
}
jsonBufPool = sync.Pool{
New: func() interface{} { return bytes.NewBuffer(make([]byte, 0, 128*1024)) },
@@ -945,8 +971,8 @@ func (w *wsListener) connect() error {
dialer := websocket.Dialer{
NetDialContext: netbind.DialContext,
HandshakeTimeout: w.config.WSHandshakeTimeout,
ReadBufferSize: 4 << 20,
WriteBufferSize: 4 << 20,
ReadBufferSize: w.config.WSReadBufferSize,
WriteBufferSize: w.config.WSWriteBufferSize,
}
conn, _, err := dialer.Dial(wsURL, header)
@@ -1141,11 +1167,17 @@ type YandexVolgaTransport struct {
}
func NewYandexVolgaTransport(docURL string, cfg transport.TransportConfig) *YandexVolgaTransport {
return NewYandexVolgaTransportWithConfig(docURL, cfg, DefaultVolgaConfig())
}
// NewYandexVolgaTransportWithConfig is NewYandexVolgaTransport with a
// resource profile, e.g. SlimVolgaConfig on a phone.
func NewYandexVolgaTransportWithConfig(docURL string, cfg transport.TransportConfig, volga VolgaConfig) *YandexVolgaTransport {
jar, _ := cookiejar.New(nil)
return &YandexVolgaTransport{
BaseTransport: transport.NewBaseTransport(cfg),
docURL: docURL,
config: DefaultVolgaConfig(),
config: volga,
stats: &VolgaStats{},
cookieJar: jar,
keepAliveStop: make(chan struct{}),
transport/yandex/yandex.go
+16 -1
View File
@@ -66,9 +66,15 @@ type DocSession struct {
func (s *DocSession) safeWrite(messageType int, data []byte) error {
s.writeMu.Lock()
defer s.writeMu.Unlock()
// A write into a half-open connection (NAT dropped it, the network
// changed) would otherwise block until the kernel gives up, minutes.
_ = s.Conn.SetWriteDeadline(time.Now().Add(docWriteTimeout))
return s.Conn.WriteMessage(messageType, data)
}
// docWriteTimeout bounds one WebSocket write to the document.
const docWriteTimeout = 20 * time.Second
type YandexDocsTransport struct {
*transport.BaseTransport
@@ -192,6 +198,9 @@ func (t *YandexDocsTransport) connectToDoc(attempt int) {
return
}
utils.Debugf("[YDOCS] fetchDocInfo failed: %v", err)
if utils.Throttled("ydocs.fetch", time.Minute) {
utils.Infof("[YDOCS] cannot open the document: %v; retrying", err)
}
t.scheduleReconnect(attempt)
return
}
@@ -264,6 +273,9 @@ func (t *YandexDocsTransport) connectToDoc(attempt int) {
_, message, err := conn.ReadMessage()
if err != nil {
utils.Debugf("[YDOCS] Read error: %v", err)
if utils.Throttled("ydocs.drop", time.Minute) {
utils.Infof("[YDOCS] connection to the document dropped: %v; reconnecting", err)
}
t.SetConnected(false)
conn.Close()
// If the session was healthy for a while, treat the next
@@ -348,8 +360,11 @@ func (t *YandexDocsTransport) keepAliveLoop() {
if session != nil && session.Conn != nil {
if err := session.safeWrite(websocket.TextMessage, []byte(keepAliveMsg)); err != nil {
utils.Debugf("[YDOCS] Keep-alive failed: %v", err)
utils.Debugf("[YDOCS] Keep-alive failed, closing the connection to reconnect: %v", err)
t.SetConnected(false)
// Close it so the reader, which may sit in ReadMessage on
// a half-open socket forever, errors out and reconnects.
_ = session.Conn.Close()
}
}
}
transport_factory.go
+7 -2
View File
@@ -31,7 +31,12 @@ func newVolgaTransport(docURL string, cfg transport.TransportConfig) (transport.
// It is the single place that knows every transport package. main.go passes
// it into manager.New, and manager calls it whenever the peer asks the exit
// to bring up an additional transport at runtime.
func transportFactory(baseCfg transport.TransportConfig) manager.Factory {
//
// isExit is this process's role: a cupsonline client must never create
// rooms of its own. (It used to be built as an exit everywhere, so a
// Session client with no or dead rooms created four new ones and waited in
// them, where the exit never came.)
func transportFactory(baseCfg transport.TransportConfig, isExit bool) manager.Factory {
return func(cfg *control.TransportConfig) (transport.Transport, error) {
if cfg == nil {
return nil, fmt.Errorf("factory: nil config")
@@ -46,7 +51,7 @@ func transportFactory(baseCfg transport.TransportConfig) manager.Factory {
case "mailru":
return mailru.NewMailruDocsTransport(cfg.URL, baseCfg), nil
case "cupsonline":
return cupsonline.NewCupsonlineTransport(cfg.URL, baseCfg, false), nil
return cupsonline.NewCupsonlineTransport(cfg.URL, baseCfg, !isExit), nil
case "oneme":
token, _ := cfg.Params["token"].(string)
uidStr, _ := cfg.Params["uid"].(string)
transport_spec.go
+7 -3
View File
@@ -65,14 +65,14 @@ func buildTransportSpecs(specs []transportSpec, urls map[string]string, extra ma
// makeRawTransport builds a raw transport from a spec without the Manager's
// factory (bootstrap path). The Manager's factory is only used for transports
// added later via SubtypeTransportStart.
func makeRawTransport(spec transportSpec, baseCfg transport.TransportConfig) (transport.Transport, error) {
func makeRawTransport(spec transportSpec, baseCfg transport.TransportConfig, isExit bool) (transport.Transport, error) {
cfg := &control.TransportConfig{
Name: spec.Name,
Type: spec.Type,
URL: spec.URL,
Params: spec.Params,
}
return transportFactory(baseCfg)(cfg)
return transportFactory(baseCfg, isExit)(cfg)
}
// registerBootstrapTransports wires every spec into the manager, and also
@@ -80,8 +80,9 @@ func makeRawTransport(spec transportSpec, baseCfg transport.TransportConfig) (tr
// can use any of them. Transports that learn their client address only when
// running go into rooms by spec name, for --share.
func registerBootstrapTransports(m *manager.Manager, specs []transportSpec, baseCfg transport.TransportConfig, secret, ctx string, rooms map[string]roomLister) error {
isExit := m.Session().IsExit()
for _, spec := range specs {
raw, err := makeRawTransport(spec, baseCfg)
raw, err := makeRawTransport(spec, baseCfg, isExit)
if err != nil {
return fmt.Errorf("%s: %w", spec.Name, err)
}
@@ -101,6 +102,9 @@ func registerBootstrapTransports(m *manager.Manager, specs []transportSpec, base
if err := m.Add(spec.Name, spec.Type, raw, spec.Priority, provider); err != nil {
return fmt.Errorf("manager add %s: %w", spec.Name, err)
}
if spec.URL != transport.ContextPlaceholder {
m.SetURL(spec.Name, spec.URL)
}
}
return nil
}
utils/logging.go
+29 -1
View File
@@ -9,6 +9,7 @@ import (
"os"
"sync"
"sync/atomic"
"time"
)
// Debug levels. Each level prints everything the ones below it do.
@@ -169,8 +170,35 @@ func emit(message string) {
// Infof always logs, regardless of debug level. Used for user-facing status
// lines (e.g. cups room open/close) that must be visible without --debug.
//
// With a log sink set (an embedding app: the Android and iOS bridges) the
// line goes there, so the app's log screen shows the same status lines as
// the CLI's output; otherwise to the standard log.
func Infof(format string, args ...interface{}) {
log.Output(2, fmt.Sprintf(format, args...))
message := fmt.Sprintf(format, args...)
logSinkMu.RLock()
sink := logSink
logSinkMu.RUnlock()
if sink != nil {
sink(message)
return
}
log.Output(2, message)
}
// throttle remembers when each Throttled key last fired.
var throttle sync.Map // key -> time.Time
// Throttled reports whether a message identified by key may be logged now:
// true at most once per every. For warnings that would otherwise repeat on
// every packet (a key mismatch drops each one).
func Throttled(key string, every time.Duration) bool {
now := time.Now()
if last, ok := throttle.Load(key); ok && now.Sub(last.(time.Time)) < every {
return false
}
throttle.Store(key, now)
return true
}
// SafeGo runs fn in a new goroutine, recovering from any panic so a crash in
utils/secret.go
+15
View File
@@ -0,0 +1,15 @@
package utils
import "unicode/utf16"
// MinSecretChars is the shortest shared secret any peer accepts.
const MinSecretChars = 16
// SecretChars is the length of a secret in characters as every OpenFlux
// client counts them: UTF-16 code units, the way Kotlin and Java measure
// String.length. Counting bytes instead (as the core once did) let the core
// accept a 10-letter Cyrillic secret (20 bytes) that Desktop and Android
// reject, so the same link worked on one device and not on another.
func SecretChars(s string) int {
return len(utf16.Encode([]rune(s)))
}