Skip to content

Fixed does not fully support fraction bits >= storage bits - 1Β #318

@PatrickHaecker

Description

@PatrickHaecker

Somehow related to #311 as it also concerns the lack of generalization, but in the other direction. If precision is really important, but range is not, we want to use all the bits to represent small numbers accepting that larger numbers, like 1, can no longer be represented.

Let b be the number of bits of the storage type and f the number of bits of the fraction. For f < b - 1 everything is fine:

using FixedPointNumbers, Unitful

julia> Fixed{Int8, 6}(1)
1.0Q1f6

julia> Fixed{Int8, 6}(1)u"m"
1.0Q1f6 m

However, for f >= b - 1 things start to break down when using units:

julia> Fixed{Int8, 7}(1)
"0.992188"

julia> Fixed{Int8, 7}(1)u"m"
ERROR: MethodError: no method matching *(::String, ::Unitful.FreeUnits{(m,), 𝐋, nothing})
The function `*` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  *(::Any, ::Any, ::Any, ::Any...)
   @ Base operators.jl:642
  *(::Missing, ::Unitful.Units)
   @ Unitful ~/.julia/packages/Unitful/8Qjvg/src/units.jl:94
  *(::LinRange, ::Unitful.Units)
   @ Unitful ~/.julia/packages/Unitful/8Qjvg/src/range.jl:143
  ...

Stacktrace:
 [1] top-level scope
   @ REPL[86]:1

The reason seems to be the additional quotes which are printed for f = 7 instead of f = 6, as can be seen above or more clearly below

julia> sprint(show, Fixed{Int8, 6}(1))
"1.0Q1f6"

julia> sprint(show, Fixed{Int8, 7}(1))
"\"0.992188\""

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions