Discuss Scratch

eaglgenes101
New to Scratch
100+ posts

Presicion of numbers?

I'm planning to do a project that is based on manipulating variables. Only thing, if any of the variable's digits fall off, the project is toast.

What is the maximum presicion of Scratch numbers? That is, how big can I make them before digits start falling off?
ProdigyZeta7
Scratcher
1000+ posts

Presicion of numbers?

About 10-14 digits, then it starts rounding…
For example, 1241231221415 becomes 1.24123122e12 and 0.000000000002321 becomes 2.321e-12.



ErnieParke
Scratcher
1000+ posts

Presicion of numbers?

I know the upper precision limit, but off the top of my head I don't know the lower one. After a few tests… it's around 10^-15. Everything past that seems to be chopped off. As for the upper limit, it's about 10^14.

By the way, out of curiosity, what project are you making?

With regards,

ErnieParke

Last edited by ErnieParke (Aug. 11, 2013 06:18:45)


eaglgenes101
New to Scratch
100+ posts

Presicion of numbers?

I did a check myself, and I found I should avoid storing more than 48 bits in a single number. The actual edge is about 53 bits, but I don't want to risk any precision.

Last edited by eaglgenes101 (Aug. 11, 2013 06:35:30)

Powered by DjangoBB