`elf_w (lookup_symbol_callback)` error code `-UNW_ENOMEM` is never bubbled up

[nevesnunes]

Environment

• libunwind version: >= 1.8.0
• architecture: x86_64
• OS: Linux

Description

On commit a35ff78, there was a refactor of elf_w (lookup_symbol), which introduced two functions: elf_w (lookup_symbol_closeness) and elf_w (lookup_symbol_callback).

Before that refactor, if users called unw_get_proc_name() with a size smaller than the symbol being retrieved, the expectation is that this would return -UNW_ENOMEM (-2).

While the logic to produce this error still exists, its value however is never bubbled up from elf_w (lookup_symbol_closeness), since we are only updating its return value when elf_w (lookup_symbol_callback) returns success:

if ((*callback) (context, &syminfo, data) == UNW_ESUCCESS)
{
  if (ret != UNW_ESUCCESS)
    ret = UNW_ESUCCESS;
}

Therefore, users can only get two possible results: UNW_ESUCCESS, or -UNW_ENOINFO, which is the default value assigned to this return variable. This seems unintended, because otherwise the logic to return -UNW_ENOMEM would have been removed. Furthermore, it is misleading to users, as they have to guess if the -UNW_ENOINFO they now get was due to an insufficient buffer size or not.

Reproduction Steps

Example:

#define UNW_LOCAL_ONLY
#include <libunwind.h>
#include <stdio.h>
#include <string.h>

void backtrace() {
  unw_cursor_t cursor;
  unw_context_t context;

  unw_getcontext(&context);
  unw_init_local(&cursor, &context);
  while (unw_step(&cursor) > 0) {
    unw_word_t offset, pc;
    unw_get_reg(&cursor, UNW_REG_IP, &pc);
    if (pc == 0) {
      break;
    }
    printf("0x%lx:", pc);

    char sym[256];
    int buf_len = 40; // This is smaller than our largest function name
    int ret = unw_get_proc_name(&cursor, sym, buf_len, &offset);
    if (ret == 0) {
      printf(" (%s+0x%lx)\n", sym, offset);
    } else {
      printf(" ret=%d", ret);
      // Let's guess if this was actually -UNW_ENOMEM
      if (strlen(sym) < buf_len) {
          printf(" (%s+0x%lx)\n", sym, offset);
      } else {
          printf("\n");
      }
    }
  }
}

void aaaa1111aaaa2222aaaa1111aaaa2222aaaa111x() {
    backtrace();
}

int main() {
    aaaa1111aaaa2222aaaa1111aaaa2222aaaa111x();
    return 0;
}

Observed output:

gcc -o libunwind_example -g libunwind_example.c -lunwind
./libunwind_example

0x55683d50d314: ret=-10 (aaaa1111aaaa2222aaaa1111aaaa2222aaaa111+0xd)
0x55683d50d325: (main+0xe)
0x7fbfdac0dca8: (__libc_start_call_main+0x78)
0x7fbfdac0dd65: (__libc_start_main_alias_2+0x85)
0x55683d50d0e1: (_start+0x21)

Expected output: ret=-2 on the function name with length 40.