SMTP Smuggling. WTF?

A few days ago, some researchers announced a novel exploit in the SMTP protocol which they have called SMTP smuggling. There has been some breathless reporting about how it bypasses security measures, will end email as we know it, dogs and cats sleeping together, and so on. However, it seems to me to be something of a tempest in a teapot for the vast majority of server operators, or at least something the vast majority of operators cannot do anything about. (Most servers running an SMTP service are not email service providers.) Anyway, their vulnerability report is here if anyone wants the details from the horse’s mouth, so to speak.

Continue reading “SMTP Smuggling. WTF?”

Ethernet Wiring Standards

I recently had cause to do some research into ethernet wiring standards as part of a troubleshooting exercise. A quick trip to the Google yielded all manner of information of varying quality passed off as absolute fact. Some of that information is clearly nonsense and I wanted to address some of that. First, I’ll start with some background and then move on with the wiring bits.

Continue reading “Ethernet Wiring Standards”

Disk Basic File Access From Assembly

Ever want to do file I/O in assembly on the CoCo using the Disk Basic environment? It turns out it isn’t as complicated as it might seem.

For those who still tinker with the old TRS-80 Color Computers, there always seems to be something to learn. Some time back, I modified Dungeons of Daggorath to run from disk and support loading and saving from disk files. However, I didn’t want to implement my own file system management code since down that road lies madness, bugs, and existentially dangerous dragons. I knew, however, that the routines I needed already existed in the Disk Basic ROM, but there were wrinkles in trying to use them, not least is the fact that they are at different addresses in the two versions of the Disk Basic ROM. Based on some discussion over on the CoCo Discord, I decided to write up a little documentation on how to use the Disk Basic (DECB) ROM routines to do file I/O from assembly language.

First, I need to define what we can do and what we can’t. DECB supports files open for input, output, or random access with fixed records. It does not support reading and writing simulaneously to the same file unless fixed length records are used. It also does not support seeking within a sequential (input or output) file. It does support byte by byte reading or writing on sequential files and accessing arbitrary fixed length records on random files. The ROM also supports deleting files and renaming files. Copying files is technically supported but the implementation is dangerous to assembly programs. With some careful coding, it should be possible to implement a seek operation for sequential files. It may also be possible to implement a read/write sequential file with careful coding. However, both of those are beyond the scope of this particular article.

Aside from initializing the whole operation, marshalling arguments, and handling the entry and exit conditions for the various ROM routines, it is also necessary to handle error conditions. When the ROM detects and error, it will vector to the usual Basic error handler which will return to the OK prompt, or, depending on what your program has done, crash the computer. A critical component of handling these routines is being able to trap the errors and return to the correct place. This is complicated by the fact that the error handler may be entered with any arbitrary amount of extra junk on the stack.

I will deal with the various problems step by step.

Required Details

First, we need some critical bits of information. Notably, the addresses of the various ROM routines that will be used. There are also a couple of other important addresses. These are listed below.

Routine Address (hex)
1.0 1.1
Get Character A176
Put Character A282
Open File C483 C48D
Close File A426
Error Vector 0192
File Number 006F
EOF Flag 0070
File Name Buffer 094C
Drive Number 00EB
File Type 0957
File ASCII Flag 0958
Default Drive 095A
Random File Record Length 097C

The file number is exactly what you would specify for the #n parameter to a Basic command. The EOF flag will be nonzero if there were no characters left to read when calling the get character routine. The error vector address is the address of the RAM hook used by the error handling routine at AC46 which is what will be used to trap errors from the ROM.

You will note that there is a single routine listed for opening a file. This routine requires a bit of work on the caller’s part to make it work properly, which will be discussed in the relevant section below.

You will also note most items list only a single address. These items are either located in the Color Basic ROM or do not vary between Disk Basic ROM versions.

Setup

First, a few things need to be set up to make everything work. The code to do this is as follows:

; Required state variables
saved_error RMB 3 ; saved error handler ram hook
disk_open RMB 2 ; open file routine address
error_handler RMB 2 ; active error handler return vector
error_stack RMB 2 ; saved stack pointer when setting error trap
; Install the error trap handler
LOADER LDA #$7E
LDX #error_hook
LDB $0191
LDU $0192
CMPU #error_hook
beq LOAD000
STA $0191
STX $0192
STB saved_error
STU saved_error+1
; Detect Disk Basic version and set routine pointers
LOAD000 LDX $C004
CMPX #$D66C
BEQ disk10
CMPX #$D75F
BEQ disk11
LDX #disk_error
BRA load010
disk10 LDX #$C483
BRA load010
disk11 LDX #$C48D
load010 STX disk_open
; Disable error handler return vector
CLR error_handler
CLR error_handler+1
RTS

The above routine does three things. First, it installs a handler in the error routine RAM hook. It saves the original error handler vector so it can return to the mainstream error handler that was previously installed if the error handling return vector is not set. If it detects that the error handler is already installed, it skips this step to prevent problems if the routine is called multiple times. Then it detects the Disk Basic ROM version by looking at the address of DSKCON stored at C004. It sets the routine pointer for opening files to the correct entry point in the ROM, or to a dummy routine that just returns error if neither known version of Disk Basic is detected. It then makes sure the error handler return vector is disabled.

Error Handling

Handling errors requires two steps. First, a trap has to be installed. Setting a trap involves saving the current stack address and a handler vector. Then the ROM routine is called. After the call, the trap must be cleared. If no error occurs, then the trap does nothing. However, if an error occurs, the trap must reset the stack, clear the error trap, and do any other state restoration required. Basically, this needs to undo whatever you need to do to call the ROM in the first place that is not routine specific. The trap vector you install will handle anything routine specific.

Without further ado, here is the error trap handling code.

disk_error
LDB #255
error_hook
LDX error_handler
BNE error_hook000
JMP saved_error
error_hook000
LDS error_stack
; do other things like restoring DP, etc.
JMP ,X
error_settrap
STX error_handler
PULS Y
STS error_stack
JMP ,Y
error_cleartrap
PSHS CC
CLR error_handler
CLR error_handler+1
PULS CC,PC

That is all deceptively simple, is it not? The routine at error_hook handles the RAM vector. If an error handler is set, it transfers control to the handler at error_hook000 after first restoring the saved stack pointer. This is necessary since the error handler may be invoked with any arbitrary extra elements on the stack. Otherwise, it transfers control to the original vector that was saved during setup. The disk_error routine simply enters the error handler with an error code of 255. Note that disk_error must not be called unless an error trap is currently active.

The error_settrap routine saves the specified handler routine address and then saves the stack pointer as though the routine was not called. This means there is no assumption that a trap was installed with this routine. error_cleartrap simply clears the vector address which disables trap handling. However, it also preserves CC which makes it suitable to be called immediately after the ROM routine returns.

The error trap routine will receive the error code in B. These will be exactly what ERNO would return on a CoCo3 except doubled. Alternatively, if the disk_error routine was invoked, B will contain 255. No other assumptions can be made about the state of the registers at this point.

Opening and Closing Files

The routine for opening a file expects a file mode in B, which will be “I” for sequential input, “O” for sequential output, or “R” or “D” for a random file. For a random file, 097C must be set to the 16 bit record length for the random file. Note that an appropriate amount of buffer space must previously have been set aside by FILES or similar. How to implement that in assembly is left as an exercise for the reader. Given the inflexibility of fixed record random files, making them work is not considered here.

Anyway, before calling the file open routine, we must set the file name at 094C (8 bytes for the filename followed by 3 bytes for the extension) which is space padded, meaning that you should fill any unused characters in either the filename or extension parts with spaces. No additional processing is done by the open routine; this is the exact form the file name will appear as on disk. It is best to restrict file names to the regular range of printable ASCII characters, but any character code may appear except the file name must not start with either character code 255 or character code 0. The drive number should be in DSKCON drive parameter at 00EB. Because this is a simple description of how to call these routines, the wrapper provided here will require the caller to do the same setup first. However, a string parsing routien for file names might be employed instead. Further, we have to pick the file number to use and store that in 006F. It might be more convenient to search for an available file number and simply use that. Doing so is beyond the scope of this document, however.

Note that for an output file, or for a random file that needs to be created, it will be created the type flags set at 0957 (type number) and 0958 (ASCII flag).

Closing a file has a lot less complication. We just put the file number to close in 006F and call the close routine.

These routines returns with C set on error and clear on success. On error, the error number will be in B. Remember that this will be double the error code that Basic would report in ERNO.

Without further ado, the file open and close wrappers.

file_open
PSHS D,X,Y,U
LDX #file_err
BSR error_settrap
JSR [disk_open]
BSR error_cleartrap
CLRA
PULS D,X,Y,U,PC
file_err COMA
STB 1,S
PULS D,X,Y,U,PC
file_close
PSHS D,X,Y,U
LDX #file_err
BSR error_settrap
JSR $A426
BSR error_cleartrap
CLRA
PULS D,X,Y,U,PC

The most notable thing above is the use of CLRA to clear carry and COMA to set carry. Also, because we save all the registers, we have to stash the error code before pulling them in the error handler.

Reading and Writing

Reading and writing is also completely straight forward. Because our error handling is slightly different, we can’t re-use the error handler from above so we’ll use a diffferent one, but it isn’t really any more complicated. The character read will be in A and the character to write will be in A. The file number to read or write will be in 006F. On error, the file will be closed and carry will be set. On EOF, 0070 will be nonzero but no error will be flagged. The code follows.

file_read
PSHS D,X,Y,U
LDX #file_ioerror
BSR error_settrap
JSR $A176
BSR error_cleartrap
STA ,s
CLRA
PULS D,X,Y,U,PC
file_write
PSHS D,X,Y,U
LDX #file_ioerror
BSR error_settrap
JSR $A282
BSR error_cleartrap
CLRA
PULS D,X,Y,U,PC
file_ioerror
STB 1,S
BSR file_close
COMA
PULS D,X,Y,U,PC

Again, note the use of COMA and CLRA to set and clear carry, and the setting of return values on the stack prior to pulling all of the registers.

Going Forward

The above should give you enough to be able to write or read a sequential file. As you can see, there are critical pieces missing for a fully featured file handling system. These include deleting files, renaming them, seeking within them, swapping between read and write on a sequential file, and handling random file records. There is a ROM entry point for deleting a file. Renaming simply requires editing the directory entry, and there are entry points for handling the steps for that. Seeking, changing mode between read and write, and random file handling likely all require direct access to the File Control Block (FCB) to be done efficiently. There is a routine to fetch the FCB pointer for a specific file number. Implementing any of these is beyond the scope of this introduction. If there is enough interest, I can cover some of these in future articles. In the mean time, I invite you to look into the Disk Basic Unravelled book for information on both the FCB structure and to find the various ROM entry points just mentioned.

Comments

You may have noticed that three of the four routines used above are in the Color Basic ROM. This is a testament to how generic the core I/O routines actually are in the Color Basic ROM. It is a bit less efficient to go through the get and put character routines or the close file routine in the Color Basic ROM instead of directly calling the Disk versions. However, it does also make things less complicated because we have fewer entry points with different addresses to deal with.

Additionally, if you do anything with the IRQ in your own code, you must make sure that when you set up your IRQ routine that you also transfer control to Disk Basic’s IRQ routine when your code finishes. If you don’t do that, then the drive motors will never shut off. You handle that similarly to how the error handler RAM hook is installed in the setup code above.

Finally, these routines could easily be spruced up a bit to be easier to use. However, my goal here was to keep the code as simple as possible to highlight the call mechanism rather than fancy coding.

Edited 2023-09-15: added more details regarding the file name format expected by the open routine and corrected some typos.

End of the Arrowverse

With just four episodes of The Flash left before the official end of the Arrowverse, I’m starting to think they might actually pull off a solid landing even if they don’t quite stick it. But based on what they just set up with Oliver’s guest spot, I’m not so sure it’s going to be quite as over as we’ve been led to believe.

Since Crisis, they’ve pussyfooted around with revealing the existence of Oliver’s new multiverse to the characters on Earth Prime. Now they have explicitly revealed it to them. That may have simply been dotting some Is and crossing some Ts but it does open up some possibilities, too.

Further, with Supermam and Lois not being officially part of the Arrowverse, yet John Diggle turned up at least once with implications of facing the same choice haunting him on other shows. I’m not sure it’s quite as separate as it seems though it seems likely it’s not on Earth prime based on next week’s previews.

It is clear that the mainline Arrowverse is ending with The Flash. There’s no way to really dispute that. But it does seem like they’re keeping things open to possibly revisit it later, say on another Earth which doesn’t have the accumulated baggage of Earth Prime. Perhaps it won’t be the CW that makes it, even. Perhaps they simply license it to another production house. Whatever happens, though, I think it’s just a little premature declare the Arrowverse completely done. I would give it a few years to see what develops.

As a final note: no, I don’t think we’re going to get a resurgence of the Arrowverse. At best we’ll get a couple of spin-off attempts along the lines of Stargirl which, let’s be honest, isn’t likely to drum up much interest overall. But it does feel like they’re keeping some options open.

Fun SQL Tricks II

Last time, I outlined some project requirements and a solution to one of the requirements, that of object IDs unique across first class objects in the system. This involved foreign key constraints, a stored function, character sets, collations, and a trigger, and also showed a use for SELECT…RETURNING. This time, I’m going to look at the other requirements specific to user objects.

Continue reading “Fun SQL Tricks II”

Fun SQL Tricks

So I recently started a new project that has a few requirements that are a little more complex than a simple web site. Exactly why these requirements are part of the project doesn’t really matter, nor does the specific nature of the project. However, along the way to implementing it, I learned a few things that will probably make the overall coding for the project simpler. These include the use of SQL triggers, stored routines, and foreign key constraints. Read on for a discussion of the first level of the fun.

Continue reading “Fun SQL Tricks”

Mr. Kenney Followup

I talked about the United Conservative Party leadership review in Alberta in my previous post. It seems Mr. Kenney probably did a similar analysis to mine. Even though he technically won the review with 51.4% of the vote, he has decided to resign. It is almost certainly the right choice since there is no possible way the in-fighting within the party will end if he stays on. I do wonder if the approval number would have been higher if the whole pandemic thing hadn’t happened.

Now the real question is who ends up becoming the new leader. Hopefully, it will be someone who the party can unite behind, but also someone who will not be off-putting to the voters at large. On the balance, the province really cannot afford an NDP repeat performance. I also hope it won’t be any of the ring leaders of the dissent over the past couple of years since I don’t want them to be rewarded. In particular, I really hope it isn’t Brian Jean who has just been chucking a wobbly since he didn’t win the party’s inaugural leadership election.

Well, whatever happens will happen. At least there should be time for the party to get organized before the next provincial election, and for the new leader to have some experience in office (assuming the winner is an actual MLA, though technically the premier doesn’t strictly have to be an MLA).

Also, hopefully whoever wins continues the good stuff that Kenney’s regime has been doing.

What I do know for certain is that whoever wins is going to find out that whatever it is they wanted to do that Kenney wasn’t doing is going to be loads harder than they think it is, if it’s even possible.

How Democracy Falls

(A followup after the leadership review results is over here.

Some people reading this will be aware of the leadership review for Jason Kenney, currently premier of Alberta and leader of the United Convervative Party in the province. This situation underscores how our democratic institutions have been failing. Even with the result of the leadership review vote due to be announced within the next few hours, it’s clear that this will not be the end of the matter, no matter what the result is.

The problem as I see it is that what appears to be a fringe element of the party led by the person who lost the original leadership election to Mr. Kenney is hell bent on defeating their perceived enemy at all costs, no matter the consequences. Indeed, reports quote sources as saying that even if the result is a landslide in Mr. Kenney’s favour, they will not accept the results. Indeed, it seems from their statements that any result that does not match with their desired outcome is untrustworthy and not to be accepted. Yet they’ll happily accept that the same process is completely trustworthy if they get the result they want. (Does that sound familiar at all? Feels like a re-run of some big democratic event from a year or so ago, but I can’t quite put my finger on it….)

Anyway, they can’t have it both ways. Either the process is trustworthy and the results are to be trusted and accepted, or the process is untrustworthy and the results are not to be trusted or accepted. It’s the same process either way. You can’t cherry pick the votes that go your way as the only trustworthy ones. At least not if you want to claim that you truly believe in the democratic will of the people being honoured.

Here’s the thing, though. Mr. Kenney has said he will accept the result of the vote. The party constitution (or rules or whatever it’s called) requires 50% plus one to pass the review. He has said he will say on if he gets 50% plus one, as he is allowed to do. He has said he expects everyone to accept the results and stop the infighting no matter which way they go. His opponents, however, seem to only be using this process as a tool to stage a coup. They’ll crow about how they won if it goes their way, but if not, they’ll crow about how it was rigged. And, to make matters worse, the mainstream media will continue to amplify their message and largely ignore or deride Kenny’s as they have been doing all along. After all, confict makes for better news, doesn’t it?

So here’s how I see things going. There are exactly two possibilities.

Kenney wins. If Kenney wins, the infighting will continue. I don’t like meme images, but if I did, I would insert one that has a caption along the lines of “infighting intensifies”. Kenney will attempt to reconcile the party, which will fail. He will likely have to kick the more disruptive members out of caucus because there’s now way to govern otherwise. He will attempt to hold on until the next regularly scheduled election and put his actual track record up against the opposition parties in the general election. I give it no better than even odds that he isn’t forced out by some other means, likely extremely shady and probably involving a frame job, within months following the review.

Kenney loses. In this case, Kenney will step aside and there will be a party leadership election. He may stay on as leader until the results of the leadership election are known. You would think that would be acceptable for his opponents since they would be getting what they want, but if it chooses this path, they will scream about him being antidemocratic by staying on while the multi-month leadership election takes place. I doubt Kenney will stand for re-election, though as I understand it, he would have the right to do so. Then, regardless of the outcome of the leadership election, the rebellious element of the party will not be united. Currently, they’re united with the goal of getting Kenney out, but beyond that, they don’t have much in common at all. The party will continue to be anything but united, and will have a very high probability of losing the next election, or at least being knocked down to minority status. (There are allegations that a vast number of new memberships were purchased with the specific goal of achieving just this result, but those haven’t been proven. Even if true, those members are probably going to be continuing to destabilize from within rather than let things settle down.)

Anyway, I expect Kenney to win the leadership review by a relatively small margin, but enough that it’s unlikely a counting error. Then, his oponents will grouse and gripe in the media for a short time and demand his resignation on various grounds, including not having met the “threshold” for continuing (he only needs 50% plus one according to the rules remember so this will all be self-serving nonsense). Then, eventually, law suits will be filed. The courts will not be amused. Nor will anyone else. Odds are they will be dismissed with prejudice after a great deal of legal brangling. Regardless of all of that, Kenney will eventually be forced to resign if there is to be any chance at actually governing or winning the next election.

On the other hand, if he loses, he has said he will step aside and we can skip all the mess in the previous paragraph. However, I don’t think his opponents have as much real support as they think they do which is why I think he will eke out a win.

The subsequent leadership election is going to be unpleasant as candidates from three camps vie for leadership: the rebels, the status quo-ers, and various “time for a change” new guys.

All of this shows neatly how a democratic system can fall. It usually takes a run of this type of behaviour to cause a fall, but if it doesn’t fall apart completely on its own, eventually someone with the charisma and resources will come along and game the chaos to their own benefit. (See many cases in history, including Julius (and, due to existence failure of the former, Augustus) a couple thousand years ago

I should be clear that this doesn’t mean we shouldn’t strive to keep our democratic systems functioning. Instead, we need to be wary of this sort of trap, which has become all too much easier to cause with the advent of algorithmic (anti)social media and such things as cancel culture.