Fix stale values in the detail struct#2881
Conversation
arch/RISCV/RISCVMapping.c
Outdated
| // memset all stalled values in the detail struct to 0 before disassembling any next instruction | ||
| void RISCV_init_cs_detail(MCInst *MI) | ||
| { | ||
| if (detail_is_set(MI)) { |
There was a problem hiding this comment.
Please invert and return early.
arch/RISCV/RISCVMapping.c
Outdated
| //for (size_t i = 0; i < ARR_SIZE(RISCV_get_detail(MI)->operands); ++i) | ||
| // RISCV_get_detail(MI)->operands[i].type = RISCV_OP_INVALID; |
There was a problem hiding this comment.
RISCV_OP_INVALID should always be 0 because it is = RISCV_OP_INVALID which is in the API.
| //for (size_t i = 0; i < ARR_SIZE(RISCV_get_detail(MI)->operands); ++i) | |
| // RISCV_get_detail(MI)->operands[i].type = RISCV_OP_INVALID; |
There was a problem hiding this comment.
You can even remove the whole assert actually.
There was a problem hiding this comment.
Yeah, if the API will never change, there is no need for assert :)
Then, should the final version be without an early return? This is all it stayed:
if (detail_is_set(MI))
memset(get_detail(MI), 0,
offsetof(cs_detail, riscv) + sizeof(cs_riscv));There was a problem hiding this comment.
It's fine like it is then :)
Just remove the assert
|
cc @moste00 |
|
@slate5 Off-topic, but I see on your profile that you are associated with the Barcelona Supercomputing Center? |
I don't know to what extent it is generally used in BSC, but I was using it as a dependency of syscall_intercept while porting it to RISC-V (the same applies for PPC and ARM ports done by BSC and RIKEN). And, GekkoFS depends on all syscall_intercept ports. |
All good. The repos you linked are enough. Thanks. |
|
Best of luck in getting funds, you deserve it ;) |
|
@moste00 Please take a look as well. |
| { | ||
| if (detail_is_set(MI)) | ||
| memset(get_detail(MI), 0, | ||
| offsetof(cs_detail, riscv) + sizeof(cs_riscv)); |
There was a problem hiding this comment.
Not sure if offsetof is warranted here, it's overly cautious to only zero out the cs_details structs up to the RISC-V member but not the rest of the strucut, the rest of the struct will probably be garbage anyway since the RISC-V member shares a memory prefix with the other members, so clearing it will likely mangle the memory start part of the other member in some complex way.
Anyway, I personally vote against this size expression and in favor of a simple sizeof(cs_detail), but I wouldn't block the PR because of it.
There was a problem hiding this comment.
Hi @moste00, by "up to the RISC-V member", do you mean including or excluding cs_riscv? memset() clears the entire cs_detail, including cs_riscv. The difference between using offsetof and only sizeof is that the total size being cleared can be much smaller (424 B vs 2216 B). It's mostly irrelevant, I just don't like making memset() sweat more than necessary XD
What do you mean by "mangle the memory start part of the other member"? The only issue I can see with offsetof would be if new elements are added to cs_detail after union, they wouldn’t be cleared.
There was a problem hiding this comment.
Oh I didn't know the real sizes, interesting!
My point was more like: the cs_detail is structured as follows, right ?
<<Beginning of cs_detail struct>>
<<<member 1>>>
<<<member 2>>>
....
<<<member N>>>
<<<Beginning of the union of arch-specific struct details>>>
<<<<arch 1 struct >>>>
<<<<arch 2 struct>>>> <---- [in parallel, sharing the same memory with arch 1 struct]
...
<<<<arch N struct>>>>
As I understand it, your size expression is clearing all the struct up to the memory representing the RISC-V details struct, is that correct ? But if so, this means there are stale bytes from the other union structs (for example, if the x86 struct is some M bytes longer than the RISC-V ones, those M bytes are stale after the clearing operation, correct?).
I thought that was due to you not wanting to clear this memory, but your argument is that the real sizes of the structs involved make this worth it, right? In that case you do have a point.
That said, this is the first time I actually see offsetof used in real code, maybe the saving is just not worth assuming it exists! Capstone is compiled for so many platforms and compilers, some of them are weird and old, and memset is mostly vector code munching through the memory range anyway.
Your call though, I have no strong preference either way. Thank you for clarifying this with numbers.
There was a problem hiding this comment.
As I understand it, your size expression is clearing all the struct up to the memory representing the RISC-V details struct, is that correct ?
Yes, those M bytes are irrelevant (if I'm not missing anything) because RV won't touch them at runtime. They will be zeroed at the start, and nothing will be written there in the RV runtime, so nothing has to be cleared.
I thought that was due to you not wanting to clear this memory, but your argument is that the real sizes of the structs involved make this worth it, right?
Haha, yes, I'm just being evil (premature optimization is the root of all evil) :)
offsetof is used in different places in Capstone already. It's part of stddef.h; in fact, I took this example of clearing like this from other architectures in Capstone. Vector instructions can make this size difference irrelevant if you have them on RV and if libc implements them. I tested on LicheePi 4A: isolated memset is 4 times faster for those two sizes, but then i used gprof on capstone, and the difference between those two functions was irrelevant (~10% if gprof can be trusted). As I said, pure evil... :)
Your call though, I have no strong preference either way. Thank you for clarifying this with numbers.
You are welcome! I clearly like things to go faster, but your approach is safer (the corner case I have mentioned: if someone ever puts a new cs_detail's member after archs union). @Rot127, can give a final verdict XD
There was a problem hiding this comment.
Anyway, I personally vote against this size expression and in favor of a simple sizeof(cs_detail), but I wouldn't block the PR because of it.
Nah, offsetof is fine. It is used like this in any other arch. And I am not sure if there is somewhere some dependency on it not being zeroed except the archs detail struct.
|
Looks good and thank you @slate5 for spotting this! I shared non-binding feedback on the size calculation you used to clear out the members, please feel free to take it into account. I think using (Sidenote: The PR description says you added tests but I can't see any test code in the changed files, this is a heads up in case you forgot to push, but it's okay if you need to merge the fix urgently as I will add the tests later.) |
|
I addressed Oh yeah, I checked that test box by mistake. I’ll add tests, and then you can merge. |
|
@moste00, aside from YAML, I added a unit test because |
Rot127
left a comment
There was a problem hiding this comment.
Nice! Thanks a lot for the test!
Wouldn't have been necessary, but I appreciate any pedantic testing!
|
Buffer overflow: Once its fixed we can merge. |
If it's fine, I can just add a fix in this PR for CAPSTONE_EXPORT
bool CAPSTONE_API cs_disasm_iter(csh ud, const uint8_t **code, size_t *size,
uint64_t *address, cs_insn *insn)
{
+ if (*size == 0)
+ return false;
+
struct cs_struct *handle;
uint16_t insn_size;
MCInst mci;
bool r;I don't know what your practice is because this is not related to this PR and will influence all architectures, so should I open a new one, or can the fix go here? |
|
Or i can place this early return guard in |
|
Yes, please open a new one. |
| { | ||
| if (detail_is_set(MI)) | ||
| memset(get_detail(MI), 0, | ||
| offsetof(cs_detail, riscv) + sizeof(cs_riscv)); |
There was a problem hiding this comment.
Anyway, I personally vote against this size expression and in favor of a simple sizeof(cs_detail), but I wouldn't block the PR because of it.
Nah, offsetof is fine. It is used like this in any other arch. And I am not sure if there is somewhere some dependency on it not being zeroed except the archs detail struct.
|
Please rebase |
7e1b572 to
3768bfd
Compare
Your checklist for this pull request
Detailed description
The bug: The detail struct (
cs_detail) isn't cleared between instructions, causing stale values. For example,op_countwould grow with each new instruction, and was only reset in rare cases when alias instructions are used but alias details are disabled (seeRISCV_LLVM_printInstruction():411).The fix: Added
RISCV_init_cs_detail()that gets called once fromRISCV_LLVM_getInstruction()before decoding each instruction.Test plan
Pseudo snippet:
Without the patch,
op_countwould grow up to the max value of uint8_t (255).Closing issues
...