People have used it for interesting things:
- Bridging connections which would otherwise be blocked by a firewall
- Tunneling through RDP (similar to an SSH tunnel)
- Exposing a localhost web server to others
Key features I put effort into:
1. The shared file is restarted every 10 MB, so it doesn't grow indefinitely.
2. Optimisations for latency & bandwidth. (800 Mbps on a Gigabit LAN. 108 Mbps if file tunneling through RDP)
3. Synchronisation between two sides (each side can be started and restarted in any order)
I'd love to hear about any weird and wonderful uses you might have for it.
Thanks, Fidel
If that is indeed the case, (I am not a Windows pr SMB expert, so perhaps you will tell me it is not the case and to be clear, I am not asserting that it is or trying to say anything negative; I am just curious and you seem like a pretty likely source to be able to answer the question, since something motivated you to write this tool)
One difference would seem at first brush is that a named pipe used for this would not need to be periodically truncated in 10 minute intervals to prevent runaway file growth. Again, not knocking what you’ve made at all, I’m genuinely curious for the answer.
Anyway, you’re right, there was no SMB involved in that readme. :-) So strange when you can’t trust your own eyes or short-term memory. Crazy.
https://www.ndchost.com/wiki/server-administration/netcat-ov...
The main trick is to do it over UDP to get speed like Aspera:
It would presume trusting the permission system over NFS, though.
good thing I needed HTTP access or else i had to find that socks5 server that's actually working again.
Do you have a good resource recommendation to learn this more?
Basically when you got a single communication channel and it's expensive to create more channels to tunnel multiple TCP streams where each connection get's its on channel, this is when you reach for the muxer.
i.e. here I probably could make it without yamux if instead I used SSH control port(socket? master?) and created many shell IO streams. but using yamux makes it cleaner. Also the server I'm working on is very buggy and you really get like 50% success rate logging in.
I suggest you download this `yamux` utility and play around with it really :)
For use on flights where they give you "messenger only" internet access.
Feature request: TCP over JPEG cat pictures on HTTP port 80
It would likely do a better job at circumventing the China firewall than existing VPNs that can be flagged as VPN protocols.
They still rate limit these flows to a very low speed.
You could just sniff the traffic for an already-logged-in MAC and clone it.
> Ping-eon would be very high.
The last 10 years of the Internet has been one long setup for this joke. Resetting the clock for 10 years.
The key to high performance as you rightly pointed out was preventing flushing. In the end, what worked best was reducing the number of writes to disk (which is the bottleneck). I did that by buffering 10-50 ms worth of TCP data, coupled with only flushing explicitly (using a large buffer so that neither BinaryWriter or FileStream flush automatically).
import socket import win32pipe import win32file import sys
BUFFER_SIZE = 4096
def create_pipe(pipe_name): return win32pipe.CreateNamedPipe(pipe_name, win32pipe.PIPE_ACCESS_DUPLEX, win32pipe.PIPE_TYPE_BYTE | win32pipe.PIPE_READMODE_BYTE | win32pipe.PIPE_WAIT, 1, BUFFER_SIZE, BUFFER_SIZE, 0, None)
def open_pipe(pipe_name): return win32file.CreateFile(pipe_name, win32file.GENERIC_READ | win32file.GENERIC_WRITE, 0, None, win32file.OPEN_EXISTING, 0, None)
def proxy_server(machine_name): server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server_socket.bind(('0.0.0.0', 8080)) server_socket.listen(1)
pipe_name_1 = r'\\' + machine_name + r'\pipe\proxy_pipe_1'
pipe_name_2 = r'\\' + machine_name + r'\pipe\proxy_pipe_2'
while True:
client_socket, _ = server_socket.accept()
pipe_1, pipe_2 = create_pipe(pipe_name_1), create_pipe(pipe_name_2)
win32pipe.ConnectNamedPipe(pipe_1, None)
win32pipe.ConnectNamedPipe(pipe_2, None)
while True:
data = client_socket.recv(BUFFER_SIZE)
if not data:
break
win32file.WriteFile(pipe_1, data)
response = win32file.ReadFile(pipe_2, BUFFER_SIZE)[1]
client_socket.send(response)
client_socket.close()
win32file.CloseHandle(pipe_1)
win32file.CloseHandle(pipe_2)
pipe_name_1 = r'\\' + server_name + r'\pipe\proxy_pipe_1_' + machine_name
pipe_name_2 = r'\\' + server_name + r'\pipe\proxy_pipe_2_' + machine_name
pipe_1, pipe_2 = open_pipe(pipe_name_1), open_pipe(pipe_name_2)
target_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
target_socket.connect((target_host, target_port))
while True:
data = win32file.ReadFile(pipe_1, BUFFER_SIZE)[1]
if not data:
break
target_socket.send(data)
response = target_socket.recv(BUFFER_SIZE)
win32file.WriteFile(pipe_2, response)
target_socket.close()
win32file.CloseHandle(pipe_1)
win32file.CloseHandle(pipe_2)
mode, machine_name = sys.argv[1], sys.argv[2]
if mode == 'server':
proxy_server(machine_name)
elif mode == 'client':
if len(sys.argv) < 4:
print("Please provide the server name or IP address for the client mode.")
sys.exit(1)
server_name = sys.argv[3]
proxy_client(server_name, machine_name)
else:
print("Invalid mode. Please use 'server' or 'client'.")
sys.exit(1)1. It gracefully supports each side of the tunnel turning on and off.
2. It accepts any number of clients, and forwards them through the tunnel.
3. It recycles the shared file.
As for the uses, see the three author listed. For example, by using a file share as transport you may evade firewall.
mkfifo /mnt/shared/connection
nc localhost 12345 > /mnt/shared/connection < /mnt/shared/connection
Also how does either side know when the file has been updated?
In the future I will implement a single file to handle both directions.
Arbitration was indeed one of the trickiest bits. Originally I pre-reallocated the full file size (10 MB). Then used an integer at the beginning of the file to signal to the other side that a block was ready. The other side repeatedly read that int, and read the corresponding part of the file. But writing twice (once for the data, once for the int) had a significant performance impact.
In the end, what worked best was not pre-allocating the file. Rather letting the file grow whenever the writer writes to it. The reader knows when data is available by doing a PeekChar() in a tight loop. It's surprisingly fast, and accurately reflects the state of the file.