Thumbnail View Resource
Source (Presentation Source), posted 2010-05-07

Added By

Avatar BillHenn on 2010-05-07

  • 4747 total views
A detailed discussion about how floating point math works in .NET and an implementation of a custom floating point type that works around issues with some floating point math.
"WPF uses double-precision floating point numbers (double in C#) in much of its public API and it uses single-precision floating point for much of its internal rendering. So floating point math is something we deal with constantly. Oddly enough, I actually knew very little about the gory details until I recently tried to write a container that had to use doubles as a key, which required working around the problems with precision in floating point math. I found the whole exercise to be very interesting, and so I’ll present what I learned here. Consider this code: if (0.8 - 0.7 == 0.1) { Console.WriteLine("Math makes sense."); } Believe it or not, the comparison will fail."
Full Link

About Presentation Source

The WPFpedia resource guide has 4 total links to this site.


There aren't any comments yet for this resource. Be the first to add one!

Add a Comment

Please log in to post comments.



Thumbnail Screenshots by Thumbshots