PyHelp - The official Scratch Python help forum | Ask, help, discuss, & more! | +800 posts! |

bigspeedfpv

yes it's by ip

cat throwing brick at camera transparent dot gif
https://bigspeed.me

MonkeyBean2

I'm working on a minecraft renderer, and eventually fully featured client in scratch, and since I'm too lazy gzip in scratch would probably be too slow, and I don't want to try connecting over cloud variables yet, I've decided to write a “simple” (more like apocalyptic spaghetti mess) proxy in python to proxy websocket connections from a js script interacting with the scratch project to the raw tcp minecraft server (and the other way around too) (btw I would still need a very similar proxy if I had chosen to use cloud variables now), but unfortunately I get an error when I call sock.recvfrom(2097151) from inside a asyncio task.

Apart from proxying, the proxy does two things:
- Split up and combine chunks of the tcp stream to get individual minecraft packets
- Handle compression

Here's the error:

Task exception was never retrieved
future: <Task finished name='Task-8' coro=<serverToClient() done, defined at proxy/proxy.py:106> exception=OSError(9, 'Bad file descriptor')>
Traceback (most recent call last):
  File "proxy/proxy.py", line 107, in serverToClient
    tcp2web(sock, websocket, statestuff)
  File "proxy/proxy.py", line 34, in tcp2web
    data, thing = sock.recvfrom(2097151)
OSError: [Errno 9] Bad file descriptor

And here's my code:
proxy/main.py:

import socket
import asyncio
from websockets.server import serve
from io import BytesIO
import lib
def handle_packet(stream, statestuff):
    if statestuff.state == 'login':
        if packet_id == 0x03: # set compression
            compression_threshhold = lib.decode_varint(stream)
            print(f'set compression, threshhold = {statestuff.compression_threshhold}')
        if packet_id == 0x02: # login success
            # print(stream.getvalue())
            print('0x02 msg login success:', stream.getvalue())
            UUID = stream.read(7)
            print(f'login success, UUID: {UUID.hex()}')
            statestuff.state = 'play'
            print('state switched to play, stopping intervention.')
def tcp2web(sock, websocket, statestuff):
    currentPacketLength = None
    carreyover = bytearray()
    while True:
        data, thing = sock.recvfrom(2097151) # errors here :/
        if len(data) == 0:
            break
        orig_stream = BytesIO(carreyover+data)
        stream_len = len(data)
        currentPacketTotalLength = None
        currentPacket = BytesIO()
        # repeat over packets
        while True:
            if currentPacketTotalLength == None:
                if orig_stream.tell() > orig_stream.getbuffer().nbytes - 3:
                    # read more data from socket, as there might not be enough data to read the entire varint
                    carreyover = orig_stream.read()
                    break
                length = lib.decode_varint(orig_stream) # get packet length
                currentPacketTotalLength = length
                currentPacket = BytesIO()
            else:
                packet_accumulated_length = currentPacket.getbuffer().nbytes # current length of recieved packet
                n_bytes_left = currentPacketTotalLength-packet_accumulated_length
                currentPacket.write(orig_stream.read(n_bytes_left))
                if n_bytes_left <= stream_len:
                    # entire packet has been read
                    # send packet over websocket (packet is stored in currentPacket)
                    orig_stream = currentPacket
                    if statestuff.compression_threshhold == None or statestuff.compression_threshhold < 0:
                        # uncompressed
                        length = packet_length
                        packet_id = lib.decode_varint(orig_stream)
                        stream = orig_stream
                    else:
                        stream, packet_id = lib.decompress_packet(orig_stream.read())
                    handle_packet(stream, statestuff)
                    websocket.send(stream)
                    if n_bytes_left < stream_len:
                        # new packet follows - continue to next iteration to read next packet
                        currentPacketTotalLength = None
                        continue
                    else:
                        break
async def web2tcp(sock, websocket, statestuff):
    async for message in websocket:
        if statestuff.compression_threshhold == None or statestuff.compression_threshhold < 0:
            # don't compress
            sock.sendall(lib.packet_noid(message))
        else:
            # do compress
            sock.sendall(lib.compressed_packet_noid(message))
class Blah:
    compression_threshhold = None
    state = 'login'
async def clientToServer(sock, websocket, statestuff):
    await web2tcp(sock, websocket, statestuff)
async def serverToClient(sock, websocket, statestuff):
    tcp2web(sock, websocket, statestuff)
async def echo(websocket):
    msg = await websocket.recv()
    print(msg)
    host, port = msg.split(':')[:2]
    if not port: port = 25565
    port = int(port)
    print(host, port)
    statestuff = Blah()
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        sock.connect((host, port))
        # print(sock.recvfrom(2097151)) # doesn't error here
        # tcp2web(sock, websocket, statestuff)
        asyncio.create_task(serverToClient(sock, websocket, statestuff))
        asyncio.create_task(clientToServer(sock, websocket, statestuff))
async def main():
    async with serve(echo, "localhost", 5823):
        await asyncio.Future()  # run forever
asyncio.run(main())

I don't think you need lib.py, as it doesn't have anything that directly interacts with the socket - it's just a library of helper functions.

9NAwRfSR-%dWFS$a$g&zBMTj2?;jvS?L5%^X%qpCX|A<*paJ=lU07c)wmt$v{TxL5

scratchGui.timeTravel.year = '2020'

applejuiceproduc

deleted

Last edited by applejuiceproduc (July 25, 2023 13:13:41)

A signature

applejuiceproduc

In a file like this:

1
Afghanistan
AF
004
034
38928346
5313081
13.65%
$500
Southern Asia
Asia
2
Aland Islands
AX
248
154
29013
15957
55.00%
$0
Northern Europe
Europe
3
Albania
AL
008
039
2877797
2296867
79.81%
$4,122
Southern Europe
Europe
4
Algeria
DZ
012
015
43851044
24532023
55.94%
$3,627
Northern Africa
Africa
5
American Samoa
AS
016
061
55191
30355
55.00%
$11,922
Polynesia
Oceania
6
Andorra
AD
020
039
77265
77751
100.63%
$36,996
Southern Europe
Europe

(only showing some of the file, it goes on for another 3,000 lines)
How would I make a python script that removes everything except the country name, the country code (AF, AX…) and the population (4 lines after the name)?

I tried this:

file_read = open("pop2.txt", "r")
data = file_read.read().split("\n")
n = 11
x = 1
for x in range(0, 100):
    try:
        del data[(n - 1) + x :: n]
        del data[(n - 2) + x :: n]
        del data[(n - 3) + x :: n]
        del data[(n - 4) + x :: n]
        del data[(n - 5) + x :: n]
        del data[(n - 7) + x :: n]
        del data[(n - 8) + x :: n]
        del data[(n - 9) + x :: n]
        del data[(n - 11) + x :: n]
        x = x + 11
    except:
        break
file2 = open("data.txt", "w")
for x in data:
    file2.write(x + "\n")

But it didn't work at all. The output was very wrong.
(I know range(0, 100) isn't enough, but it still didn't even do the first ones right.)

Last edited by applejuiceproduc (July 25, 2023 14:52:14)

A signature

rishi272011

applejuiceproduc wrote:

In a file like this:

1
Afghanistan
AF
004
034
38928346
5313081
13.65%
$500
Southern Asia
Asia
2
Aland Islands
AX
248
154
29013
15957
55.00%
$0
Northern Europe
Europe
3
Albania
AL
008
039
2877797
2296867
79.81%
$4,122
Southern Europe
Europe
4
Algeria
DZ
012
015
43851044
24532023
55.94%
$3,627
Northern Africa
Africa
5
American Samoa
AS
016
061
55191
30355
55.00%
$11,922
Polynesia
Oceania
6
Andorra
AD
020
039
77265
77751
100.63%
$36,996
Southern Europe
Europe

(only showing some of the file, it goes on for another 3,000 lines)
How would I make a python script that removes everything except the country name, the country code (AF, AX…) and the population (4 lines after the name)?

try this:

data = []
previous_out = "_"
with open("pop2.txt", "r") as f:
    for lcv in range(0, 10000): #Change this to whatever is required
        f.readline() #skip number
        name = f.readline().strip("\n")
        code = f.readline().strip("\n")
        for i in range(2):
            f.readline()
        population = f.readline().strip("\n")
        for i in range(5):
            f.readline()
        
        #store the data
        data.append(name)
        data.append(code)
        data.append(population)
        new_out = "*" * int((lcv / 500))
        if not (new_out == previous_out):
            print(new_out)
            previous_out = new_out
print("length: ", len(data))
with open("pop2.txt", "w") as f:
    for x in data:
        f.write(x + "\n")

Last edited by rishi272011 (July 25, 2023 15:49:14)

Air_heads

MonkeyBean2 wrote:

I'm working on a minecraft renderer, and eventually fully featured client in scratch, and since I'm too lazy gzip in scratch would probably be too slow, and I don't want to try connecting over cloud variables yet, I've decided to write a “simple” (more like apocalyptic spaghetti mess) proxy in python to proxy websocket connections from a js script interacting with the scratch project to the raw tcp minecraft server (and the other way around too) (btw I would still need a very similar proxy if I had chosen to use cloud variables now), but unfortunately I get an error when I call sock.recvfrom(2097151) from inside a asyncio task.

Apart from proxying, the proxy does two things:
- Split up and combine chunks of the tcp stream to get individual minecraft packets
- Handle compression

Here's the error:

Task exception was never retrieved
future: <Task finished name='Task-8' coro=<serverToClient() done, defined at proxy/proxy.py:106> exception=OSError(9, 'Bad file descriptor')>
Traceback (most recent call last):
  File "proxy/proxy.py", line 107, in serverToClient
    tcp2web(sock, websocket, statestuff)
  File "proxy/proxy.py", line 34, in tcp2web
    data, thing = sock.recvfrom(2097151)
OSError: [Errno 9] Bad file descriptor

And here's my code:
proxy/main.py:

import socket
import asyncio
from websockets.server import serve
from io import BytesIO
import lib
def handle_packet(stream, statestuff):
    if statestuff.state == 'login':
        if packet_id == 0x03: # set compression
            compression_threshhold = lib.decode_varint(stream)
            print(f'set compression, threshhold = {statestuff.compression_threshhold}')
        if packet_id == 0x02: # login success
            # print(stream.getvalue())
            print('0x02 msg login success:', stream.getvalue())
            UUID = stream.read(7)
            print(f'login success, UUID: {UUID.hex()}')
            statestuff.state = 'play'
            print('state switched to play, stopping intervention.')
def tcp2web(sock, websocket, statestuff):
    currentPacketLength = None
    carreyover = bytearray()
    while True:
        data, thing = sock.recvfrom(2097151) # errors here :/
        if len(data) == 0:
            break
        orig_stream = BytesIO(carreyover+data)
        stream_len = len(data)
        currentPacketTotalLength = None
        currentPacket = BytesIO()
        # repeat over packets
        while True:
            if currentPacketTotalLength == None:
                if orig_stream.tell() > orig_stream.getbuffer().nbytes - 3:
                    # read more data from socket, as there might not be enough data to read the entire varint
                    carreyover = orig_stream.read()
                    break
                length = lib.decode_varint(orig_stream) # get packet length
                currentPacketTotalLength = length
                currentPacket = BytesIO()
            else:
                packet_accumulated_length = currentPacket.getbuffer().nbytes # current length of recieved packet
                n_bytes_left = currentPacketTotalLength-packet_accumulated_length
                currentPacket.write(orig_stream.read(n_bytes_left))
                if n_bytes_left <= stream_len:
                    # entire packet has been read
                    # send packet over websocket (packet is stored in currentPacket)
                    orig_stream = currentPacket
                    if statestuff.compression_threshhold == None or statestuff.compression_threshhold < 0:
                        # uncompressed
                        length = packet_length
                        packet_id = lib.decode_varint(orig_stream)
                        stream = orig_stream
                    else:
                        stream, packet_id = lib.decompress_packet(orig_stream.read())
                    handle_packet(stream, statestuff)
                    websocket.send(stream)
                    if n_bytes_left < stream_len:
                        # new packet follows - continue to next iteration to read next packet
                        currentPacketTotalLength = None
                        continue
                    else:
                        break
async def web2tcp(sock, websocket, statestuff):
    async for message in websocket:
        if statestuff.compression_threshhold == None or statestuff.compression_threshhold < 0:
            # don't compress
            sock.sendall(lib.packet_noid(message))
        else:
            # do compress
            sock.sendall(lib.compressed_packet_noid(message))
class Blah:
    compression_threshhold = None
    state = 'login'
async def clientToServer(sock, websocket, statestuff):
    await web2tcp(sock, websocket, statestuff)
async def serverToClient(sock, websocket, statestuff):
    tcp2web(sock, websocket, statestuff)
async def echo(websocket):
    msg = await websocket.recv()
    print(msg)
    host, port = msg.split(':')[:2]
    if not port: port = 25565
    port = int(port)
    print(host, port)
    statestuff = Blah()
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        sock.connect((host, port))
        # print(sock.recvfrom(2097151)) # doesn't error here
        # tcp2web(sock, websocket, statestuff)
        asyncio.create_task(serverToClient(sock, websocket, statestuff))
        asyncio.create_task(clientToServer(sock, websocket, statestuff))
async def main():
    async with serve(echo, "localhost", 5823):
        await asyncio.Future()  # run forever
asyncio.run(main())

I don't think you need lib.py, as it doesn't have anything that directly interacts with the socket - it's just a library of helper functions.

Maybe use a await keyword? I never really used asyncio before so idk…

Air_heads

MonkeyBean2 wrote:

I'm working on a minecraft renderer, and eventually fully featured client in scratch, and since I'm too lazy gzip in scratch would probably be too slow, and I don't want to try connecting over cloud variables yet, I've decided to write a “simple” (more like apocalyptic spaghetti mess) proxy in python to proxy websocket connections from a js script interacting with the scratch project to the raw tcp minecraft server (and the other way around too) (btw I would still need a very similar proxy if I had chosen to use cloud variables now), but unfortunately I get an error when I call sock.recvfrom(2097151) from inside a asyncio task.

Apart from proxying, the proxy does two things:
- Split up and combine chunks of the tcp stream to get individual minecraft packets
- Handle compression

Here's the error:

Task exception was never retrieved
future: <Task finished name='Task-8' coro=<serverToClient() done, defined at proxy/proxy.py:106> exception=OSError(9, 'Bad file descriptor')>
Traceback (most recent call last):
  File "proxy/proxy.py", line 107, in serverToClient
    tcp2web(sock, websocket, statestuff)
  File "proxy/proxy.py", line 34, in tcp2web
    data, thing = sock.recvfrom(2097151)
OSError: [Errno 9] Bad file descriptor

And here's my code:
proxy/main.py:

import socket
import asyncio
from websockets.server import serve
from io import BytesIO
import lib
def handle_packet(stream, statestuff):
    if statestuff.state == 'login':
        if packet_id == 0x03: # set compression
            compression_threshhold = lib.decode_varint(stream)
            print(f'set compression, threshhold = {statestuff.compression_threshhold}')
        if packet_id == 0x02: # login success
            # print(stream.getvalue())
            print('0x02 msg login success:', stream.getvalue())
            UUID = stream.read(7)
            print(f'login success, UUID: {UUID.hex()}')
            statestuff.state = 'play'
            print('state switched to play, stopping intervention.')
def tcp2web(sock, websocket, statestuff):
    currentPacketLength = None
    carreyover = bytearray()
    while True:
        data, thing = sock.recvfrom(2097151) # errors here :/
        if len(data) == 0:
            break
        orig_stream = BytesIO(carreyover+data)
        stream_len = len(data)
        currentPacketTotalLength = None
        currentPacket = BytesIO()
        # repeat over packets
        while True:
            if currentPacketTotalLength == None:
                if orig_stream.tell() > orig_stream.getbuffer().nbytes - 3:
                    # read more data from socket, as there might not be enough data to read the entire varint
                    carreyover = orig_stream.read()
                    break
                length = lib.decode_varint(orig_stream) # get packet length
                currentPacketTotalLength = length
                currentPacket = BytesIO()
            else:
                packet_accumulated_length = currentPacket.getbuffer().nbytes # current length of recieved packet
                n_bytes_left = currentPacketTotalLength-packet_accumulated_length
                currentPacket.write(orig_stream.read(n_bytes_left))
                if n_bytes_left <= stream_len:
                    # entire packet has been read
                    # send packet over websocket (packet is stored in currentPacket)
                    orig_stream = currentPacket
                    if statestuff.compression_threshhold == None or statestuff.compression_threshhold < 0:
                        # uncompressed
                        length = packet_length
                        packet_id = lib.decode_varint(orig_stream)
                        stream = orig_stream
                    else:
                        stream, packet_id = lib.decompress_packet(orig_stream.read())
                    handle_packet(stream, statestuff)
                    websocket.send(stream)
                    if n_bytes_left < stream_len:
                        # new packet follows - continue to next iteration to read next packet
                        currentPacketTotalLength = None
                        continue
                    else:
                        break
async def web2tcp(sock, websocket, statestuff):
    async for message in websocket:
        if statestuff.compression_threshhold == None or statestuff.compression_threshhold < 0:
            # don't compress
            sock.sendall(lib.packet_noid(message))
        else:
            # do compress
            sock.sendall(lib.compressed_packet_noid(message))
class Blah:
    compression_threshhold = None
    state = 'login'
async def clientToServer(sock, websocket, statestuff):
    await web2tcp(sock, websocket, statestuff)
async def serverToClient(sock, websocket, statestuff):
    tcp2web(sock, websocket, statestuff)
async def echo(websocket):
    msg = await websocket.recv()
    print(msg)
    host, port = msg.split(':')[:2]
    if not port: port = 25565
    port = int(port)
    print(host, port)
    statestuff = Blah()
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        sock.connect((host, port))
        # print(sock.recvfrom(2097151)) # doesn't error here
        # tcp2web(sock, websocket, statestuff)
        asyncio.create_task(serverToClient(sock, websocket, statestuff))
        asyncio.create_task(clientToServer(sock, websocket, statestuff))
async def main():
    async with serve(echo, "localhost", 5823):
        await asyncio.Future()  # run forever
asyncio.run(main())

I don't think you need lib.py, as it doesn't have anything that directly interacts with the socket - it's just a library of helper functions.

“A socket can not be reused once the connection is closed, i.e. you can not call socket.connect() on a closed socket.” - Random stackoverflow post | You need to create a new socket every time the connection is closed.

-RabbitWorld-

Is it possible to use python like php and store data? Make a login system?

♥ SDS Manager
♥ 6 Featured Studios
♥ FPC #517
♥ SWC Reviewer

davidtheplatform

-RabbitWorld- wrote:
Is it possible to use python like php and store data? Make a login system?

Yes, you can open files with open(file name).
More info: https://www.w3schools.com/python/python_file_handling.asp

Generation 4: the first time you see this copy and paste it on top of your sig in the scratch forums and increase generation by 1. Social experiment.

LoIdesMio

Air_heads wrote:

MonkeyBean2 wrote:

I'm working on a minecraft renderer, and eventually fully featured client in scratch, and since I'm too lazy gzip in scratch would probably be too slow, and I don't want to try connecting over cloud variables yet, I've decided to write a “simple” (more like apocalyptic spaghetti mess) proxy in python to proxy websocket connections from a js script interacting with the scratch project to the raw tcp minecraft server (and the other way around too) (btw I would still need a very similar proxy if I had chosen to use cloud variables now), but unfortunately I get an error when I call sock.recvfrom(2097151) from inside a asyncio task.

Apart from proxying, the proxy does two things:
- Split up and combine chunks of the tcp stream to get individual minecraft packets
- Handle compression

Here's the error:

Task exception was never retrieved
future: <Task finished name='Task-8' coro=<serverToClient() done, defined at proxy/proxy.py:106> exception=OSError(9, 'Bad file descriptor')>
Traceback (most recent call last):
  File "proxy/proxy.py", line 107, in serverToClient
    tcp2web(sock, websocket, statestuff)
  File "proxy/proxy.py", line 34, in tcp2web
    data, thing = sock.recvfrom(2097151)
OSError: [Errno 9] Bad file descriptor

And here's my code:
proxy/main.py:

import socket
import asyncio
from websockets.server import serve
from io import BytesIO
import lib
def handle_packet(stream, statestuff):
    if statestuff.state == 'login':
        if packet_id == 0x03: # set compression
            compression_threshhold = lib.decode_varint(stream)
            print(f'set compression, threshhold = {statestuff.compression_threshhold}')
        if packet_id == 0x02: # login success
            # print(stream.getvalue())
            print('0x02 msg login success:', stream.getvalue())
            UUID = stream.read(7)
            print(f'login success, UUID: {UUID.hex()}')
            statestuff.state = 'play'
            print('state switched to play, stopping intervention.')
def tcp2web(sock, websocket, statestuff):
    currentPacketLength = None
    carreyover = bytearray()
    while True:
        data, thing = sock.recvfrom(2097151) # errors here :/
        if len(data) == 0:
            break
        orig_stream = BytesIO(carreyover+data)
        stream_len = len(data)
        currentPacketTotalLength = None
        currentPacket = BytesIO()
        # repeat over packets
        while True:
            if currentPacketTotalLength == None:
                if orig_stream.tell() > orig_stream.getbuffer().nbytes - 3:
                    # read more data from socket, as there might not be enough data to read the entire varint
                    carreyover = orig_stream.read()
                    break
                length = lib.decode_varint(orig_stream) # get packet length
                currentPacketTotalLength = length
                currentPacket = BytesIO()
            else:
                packet_accumulated_length = currentPacket.getbuffer().nbytes # current length of recieved packet
                n_bytes_left = currentPacketTotalLength-packet_accumulated_length
                currentPacket.write(orig_stream.read(n_bytes_left))
                if n_bytes_left <= stream_len:
                    # entire packet has been read
                    # send packet over websocket (packet is stored in currentPacket)
                    orig_stream = currentPacket
                    if statestuff.compression_threshhold == None or statestuff.compression_threshhold < 0:
                        # uncompressed
                        length = packet_length
                        packet_id = lib.decode_varint(orig_stream)
                        stream = orig_stream
                    else:
                        stream, packet_id = lib.decompress_packet(orig_stream.read())
                    handle_packet(stream, statestuff)
                    websocket.send(stream)
                    if n_bytes_left < stream_len:
                        # new packet follows - continue to next iteration to read next packet
                        currentPacketTotalLength = None
                        continue
                    else:
                        break
async def web2tcp(sock, websocket, statestuff):
    async for message in websocket:
        if statestuff.compression_threshhold == None or statestuff.compression_threshhold < 0:
            # don't compress
            sock.sendall(lib.packet_noid(message))
        else:
            # do compress
            sock.sendall(lib.compressed_packet_noid(message))
class Blah:
    compression_threshhold = None
    state = 'login'
async def clientToServer(sock, websocket, statestuff):
    await web2tcp(sock, websocket, statestuff)
async def serverToClient(sock, websocket, statestuff):
    tcp2web(sock, websocket, statestuff)
async def echo(websocket):
    msg = await websocket.recv()
    print(msg)
    host, port = msg.split(':')[:2]
    if not port: port = 25565
    port = int(port)
    print(host, port)
    statestuff = Blah()
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        sock.connect((host, port))
        # print(sock.recvfrom(2097151)) # doesn't error here
        # tcp2web(sock, websocket, statestuff)
        asyncio.create_task(serverToClient(sock, websocket, statestuff))
        asyncio.create_task(clientToServer(sock, websocket, statestuff))
async def main():
    async with serve(echo, "localhost", 5823):
        await asyncio.Future()  # run forever
asyncio.run(main())

I don't think you need lib.py, as it doesn't have anything that directly interacts with the socket - it's just a library of helper functions.

“A socket can not be reused once the connection is closed, i.e. you can not call socket.connect() on a closed socket.” - Random stackoverflow post | You need to create a new socket every time the connection is closed.

Or just not close the connection p.s i dont know what im saying but it sounded right enough

Last edited by LoIdesMio (Aug. 1, 2023 22:54:30)

I make cloud Games! Using Scratchattach by @TimMcCool!

ajskateboarder

-RabbitWorld- wrote:
Is it possible to use python like php and store data? Make a login system?

If you want to use a markup language that works similar to PHP, try Flask (a Python web framework) with Jinja (the markup language).

Here's a primer on it: https://realpython.com/primer-on-jinja-templating/

Air_heads

LoIdesMio wrote:

Air_heads wrote:

MonkeyBean2 wrote:

I'm working on a minecraft renderer, and eventually fully featured client in scratch, and since I'm too lazy gzip in scratch would probably be too slow, and I don't want to try connecting over cloud variables yet, I've decided to write a “simple” (more like apocalyptic spaghetti mess) proxy in python to proxy websocket connections from a js script interacting with the scratch project to the raw tcp minecraft server (and the other way around too) (btw I would still need a very similar proxy if I had chosen to use cloud variables now), but unfortunately I get an error when I call sock.recvfrom(2097151) from inside a asyncio task.

Apart from proxying, the proxy does two things:
- Split up and combine chunks of the tcp stream to get individual minecraft packets
- Handle compression

Here's the error:

Task exception was never retrieved
future: <Task finished name='Task-8' coro=<serverToClient() done, defined at proxy/proxy.py:106> exception=OSError(9, 'Bad file descriptor')>
Traceback (most recent call last):
  File "proxy/proxy.py", line 107, in serverToClient
    tcp2web(sock, websocket, statestuff)
  File "proxy/proxy.py", line 34, in tcp2web
    data, thing = sock.recvfrom(2097151)
OSError: [Errno 9] Bad file descriptor

And here's my code:
proxy/main.py:

import socket
import asyncio
from websockets.server import serve
from io import BytesIO
import lib
def handle_packet(stream, statestuff):
    if statestuff.state == 'login':
        if packet_id == 0x03: # set compression
            compression_threshhold = lib.decode_varint(stream)
            print(f'set compression, threshhold = {statestuff.compression_threshhold}')
        if packet_id == 0x02: # login success
            # print(stream.getvalue())
            print('0x02 msg login success:', stream.getvalue())
            UUID = stream.read(7)
            print(f'login success, UUID: {UUID.hex()}')
            statestuff.state = 'play'
            print('state switched to play, stopping intervention.')
def tcp2web(sock, websocket, statestuff):
    currentPacketLength = None
    carreyover = bytearray()
    while True:
        data, thing = sock.recvfrom(2097151) # errors here :/
        if len(data) == 0:
            break
        orig_stream = BytesIO(carreyover+data)
        stream_len = len(data)
        currentPacketTotalLength = None
        currentPacket = BytesIO()
        # repeat over packets
        while True:
            if currentPacketTotalLength == None:
                if orig_stream.tell() > orig_stream.getbuffer().nbytes - 3:
                    # read more data from socket, as there might not be enough data to read the entire varint
                    carreyover = orig_stream.read()
                    break
                length = lib.decode_varint(orig_stream) # get packet length
                currentPacketTotalLength = length
                currentPacket = BytesIO()
            else:
                packet_accumulated_length = currentPacket.getbuffer().nbytes # current length of recieved packet
                n_bytes_left = currentPacketTotalLength-packet_accumulated_length
                currentPacket.write(orig_stream.read(n_bytes_left))
                if n_bytes_left <= stream_len:
                    # entire packet has been read
                    # send packet over websocket (packet is stored in currentPacket)
                    orig_stream = currentPacket
                    if statestuff.compression_threshhold == None or statestuff.compression_threshhold < 0:
                        # uncompressed
                        length = packet_length
                        packet_id = lib.decode_varint(orig_stream)
                        stream = orig_stream
                    else:
                        stream, packet_id = lib.decompress_packet(orig_stream.read())
                    handle_packet(stream, statestuff)
                    websocket.send(stream)
                    if n_bytes_left < stream_len:
                        # new packet follows - continue to next iteration to read next packet
                        currentPacketTotalLength = None
                        continue
                    else:
                        break
async def web2tcp(sock, websocket, statestuff):
    async for message in websocket:
        if statestuff.compression_threshhold == None or statestuff.compression_threshhold < 0:
            # don't compress
            sock.sendall(lib.packet_noid(message))
        else:
            # do compress
            sock.sendall(lib.compressed_packet_noid(message))
class Blah:
    compression_threshhold = None
    state = 'login'
async def clientToServer(sock, websocket, statestuff):
    await web2tcp(sock, websocket, statestuff)
async def serverToClient(sock, websocket, statestuff):
    tcp2web(sock, websocket, statestuff)
async def echo(websocket):
    msg = await websocket.recv()
    print(msg)
    host, port = msg.split(':')[:2]
    if not port: port = 25565
    port = int(port)
    print(host, port)
    statestuff = Blah()
    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
        sock.connect((host, port))
        # print(sock.recvfrom(2097151)) # doesn't error here
        # tcp2web(sock, websocket, statestuff)
        asyncio.create_task(serverToClient(sock, websocket, statestuff))
        asyncio.create_task(clientToServer(sock, websocket, statestuff))
async def main():
    async with serve(echo, "localhost", 5823):
        await asyncio.Future()  # run forever
asyncio.run(main())

I don't think you need lib.py, as it doesn't have anything that directly interacts with the socket - it's just a library of helper functions.

“A socket can not be reused once the connection is closed, i.e. you can not call socket.connect() on a closed socket.” - Random stackoverflow post | You need to create a new socket every time the connection is closed.

Or just not close the connection p.s i dont know what im saying but it sounded right enough

1. You probably need to close it when a user leaves the project, so that the server doesn’t error out.
2. You probably would need to close it for testing purposes.

wvzack

I just got a raspberry pi and tried downloading scratch attach with pip install -U scratchattach it worked but when i try importing it it says module not found. Thank!

Likes: Programing (python scratch Lua c++ I am bad at c++ a) Aiviation (B787 Queen of the sky F35) Mountain Biking and Messing around with computers (not a gamer at all)
My projects I am most happy with also the few “polished games I have:

mybearworld

wvzack wrote:
I just got a raspberry pi and tried downloading scratch attach with pip install -U scratchattach it worked but when i try importing it it says module not found. Thank!

The -U means upgrade. You should run it without the -U if you're installing it for the first time.

Signatures are the only place where assets links still work.

wvzack

mybearworld wrote:
wvzack wrote:
I just got a raspberry pi and tried downloading scratch attach with pip install -U scratchattach it worked but when i try importing it it says module not found. Thank!
The -U means upgrade. You should run it without the -U if you're installing it for the first time.

ok i will try. still doesn't work.

Last edited by wvzack (Aug. 24, 2023 12:13:49)

Likes: Programing (python scratch Lua c++ I am bad at c++ a) Aiviation (B787 Queen of the sky F35) Mountain Biking and Messing around with computers (not a gamer at all)
My projects I am most happy with also the few “polished games I have:

mybearworld

wvzack wrote:
(#915)
mybearworld wrote:
wvzack wrote:
I just got a raspberry pi and tried downloading scratch attach with pip install -U scratchattach it worked but when i try importing it it says module not found. Thank!
The -U means upgrade. You should run it without the -U if you're installing it for the first time.
ok i will try. still doesn't work.

Can you copy and paste your prompt and the output?

Signatures are the only place where assets links still work.

mybearworld

wvzack wrote:
(#917)
mybearworld wrote:
Can you copy and paste your prompt and the output?
No i am using a vnc i will take screenshot.
-snip-

Hmm. Can you try using `pip3 install scratchattach` (instead of just `pip`)?

Signatures are the only place where assets links still work.

wvzack

mybearworld wrote:
wvzack wrote:
(#917)
mybearworld wrote:
Can you copy and paste your prompt and the output?
No i am using a vnc i will take screenshot.
-snip-
Hmm. Can you try using `pip3 install scratchattach` (instead of just `pip`)?

Ok.

Likes: Programing (python scratch Lua c++ I am bad at c++ a) Aiviation (B787 Queen of the sky F35) Mountain Biking and Messing around with computers (not a gamer at all)
My projects I am most happy with also the few “polished games I have:

wvzack

mybearworld wrote:
wvzack wrote:
(#917)
mybearworld wrote:
Can you copy and paste your prompt and the output?
No i am using a vnc i will take screenshot.
-snip-
Hmm. Can you try using `pip3 install scratchattach` (instead of just `pip`)?

It worked thx so much!

Likes: Programing (python scratch Lua c++ I am bad at c++ a) Aiviation (B787 Queen of the sky F35) Mountain Biking and Messing around with computers (not a gamer at all)
My projects I am most happy with also the few “polished games I have:

wvzack

My code that should make a newline is not working

time = datetime.now()
timestring = now.strftime("%d/%m/%Y %H:%M:%S")
notes = project.notes
project.set_notes(notes, + "/n", + timestring)

it returns

Traceback (most recent call last):
  File "/home/pi/Python scripts/codestreamdowntime.py", line 12, in <module>
    project.set_notes(notes, + "/n", + dt_string)
TypeError: bad operand type for unary +: 'str'

Likes: Programing (python scratch Lua c++ I am bad at c++ a) Aiviation (B787 Queen of the sky F35) Mountain Biking and Messing around with computers (not a gamer at all)
My projects I am most happy with also the few “polished games I have:

Discuss Scratch