Hi,

How do I append a byte to a bytes?

(bytes-append bs (bytes b))

ah thanks.

> (char->integer (read-char (open-input-bytes (bytes 255)))) 65533

Why doesn’t it return 255?

maybe something to do with numeric conversions

255 is all 1’s in binary, which is –1 when interpreted as two’s complement (in 8 bits)

well, that still doesn’t explain why it doesn’t return 255.

So how can I make sure to read the 255 correctly?

what does the read-char expression evaluate to?

#\�

did a little digging: the unicode char with codepoint 255 is not represented with a single byte in utf8, it’s represented with two bytes - so a bytestring with just one byte of 255 is not a valid utf 8 string (which is the encoding that open-input-bytes uses)

and I think 65533 is the codepoint for an unrecognized character

Ah that makes sense.

> (bytes->list (string->bytes/utf-8 (string (integer->char 255))))
'(195 191)

so say if I have a bytes 97 109 101 58 32 255 249. How do I read the chars before 255 correctly?

do you know that it’s utf 8 encoded?

I guess it’s not then. Actually, this is telnet data, in which 255 is a special cmd, instead of data.

i have no idea how telnet works there ¯_(ツ)_/¯

so it’s normal utf8 chars, with some cmd bytes.

well, you don’t need to know…

yeah, that’s fine. I guess I’ll just have to read by byte, and then convert the utf8 bytes into chars.

>All data octets except 0xff are transmitted over Telnet as is. (0xff, or 255 in decimal, is the IAC byte (Interpret As Command) which signals that the next byte is a telnet command. The command to insert 0xff into the stream is 0xff, so 0xff need to be escaped by doubling it when sending data over the telnet protocol.)

hmm so yeah, guess you gotta turn your telnet-stream into a telnet-data-stream by ripping out each 255 and the byte after it, unless its a command that changes the meaning of the data like that “insert 255 into the stream” command

In my case it’s fine to just strip off all the 255s

do you have access to some reusable code that decodes telnet streams? I think if you just strip out only the 255s you’ll still have leftover command bytes that will mess up utf8 decoding

Yeah you are right. I don’t have that. I was actually looking for some telnet lib in racket, but didn’t find any.

also ideally, I think I should have a function for tcp-input-port -> telnet-data-port , but I don’t know how to write this function for now.

and I suspect writing a telnet implementation is not how you’d like to spend your wednesday evening

there might be something at http://pkgs.racket-lang.org\|pkgs.racket-lang.org

hahaha. I don’t need a full telnet protocol. I just need to ignore all the cmds.

Do you have an idea how to wrap a tcp port into another port that skips the cmd bytes?

@markx it’s tricky because some of the command bytes shouldn’t be stripped, for instance <cmd> 255 shouldn’t be stripped and should be replaced with 255 because command 255 means “insert byte 255 into the data stream”

I suspect there are other commands that change the data stream and if you ignore those commands instead of actually interpreting them, you’ll garble the text output because bytes will be missing or incorrect during utf8 decoding

is it possible for you to change the thing that’s actually sending these bytes to do something else? like open a tcp connection or a unix domain socket or something?

@notjack Sorry I don’t understand. Do you mean to change the thing not to use telnet?

I’m writing a MUD client, and don’t have control on the server… So I can’t change that. But it’s really fine. I can ignore the cmds for now. I can add support for cmds later if necessary.

currently my problem is that, I have the logic of skipping cmd bytes and normal data together, but I really want to separate them. Like I said, I think ideally I can make a decorator, that wraps the tcp input port and only output data bytes. But I’m not sure how to create such a decorator.

@markx ah yeah I meant change the thing to not use telnet, but if it’s not your server then that’s a no-go

as for a decorator, I think you could make that by using make-pipe

basically, you take your raw input and you make a pipe, then you spawn a thread that reads the raw input, decodes it into command-less bytes, then writes it into the output end of the pipe

then the input end of the pipe becomes your new wrapper input port

@notjack Ah sounds good. If I need to spawn a thread, I guess I can also use a channel instead of a pipe? What’s the advantage of using a pipe then?

the make-pipe procedure returns an input-port? and an output-port? so you don’t have to change any code you’ve written that assumes its reading data from an input port

but other than that, they’re pretty similar approaches

you can think of make-pipe as implementing an input-output port pair using a buffered async channel under the hood

oh, I see. I’ll try that. thanks @notjack So much for a Wednesday night!

don’t get too caught up :) sleep is good

@markx When I learned about pipes, it wasn’t obvious to me how to use them. I didn’t appreciate how to use Racket threads. Here’s an example: http://pasterack.org/pastes/22031

btw the (λ () __) can also be written as (lambda () __) or (thunk __), as you prefer.

@greg wow! did you just hand write this example for me? this is awesome!

Well I didn’t mean to do it all for you, and take the fun away. Maybe I should have changed the example more. :slightly_smiling_face:

It’s just, I found it freaky at first. I just start a thread and it runs…? That’s the part I wanted to emphasize.

haha, I’m actually comfortable with threads. I came with some Go experience, which also uses CSP for concurrency, so they are pretty similar.

Ah, cool. Very similar.

The hardest part of racket for me is maybe how to organize the code. I mean, I don’t have enough functional programming experience, and that’s what I’m trying to learn from racket.

oh, and how to setup emacs. haha because I’m a vim user.

Yeah, that takes awhile. For me coming from C, the “all the parentheses” wasn’t such a big deal, really. It was more, “How do I get anything done if I can’t loop and mutate a variable??”

I actually don’t understand why “all the parentheses” would be a problem at all. I see them the same as whitespace… Just a symbol to separate other tokens and make up the syntax.

yeah that, and what global vars and parameters should I use. If I use them too much, it feels wrong.

It’s more of a style and design problem.

funnily enough I find that people coming to racket from java don’t have the “how do I do things without loops + mutation” problem as much, assuming they’ve gotten used to doing stuff with java 8 streams