Overclock.net › Forums › Software, Programming and Coding › Coding and Programming › Floating Point Comparisons in VB.NET
New Posts  All Forums:Forum Nav:

Floating Point Comparisons in VB.NET  

post #1 of 3
Thread Starter 
So, I've got a program that does spherical <-> cartesian and cylindrical <-> cartesian transformations (the binary's here) but the main issue I'm having is how to implement a reliable "less than some small number" test so I can force results on the order of 10^-10 to 10^-16 to just be rendered as zero, since physically, the difference would be practically un-noticeable.

But what should be simple tests such as "absolutely less than 0.000001" ( < 0.000001 ) don't seem to give consistent results.

Help?

I have an answer, thanks. smile.gif
Edited by Quantum Reality - 4/25/14 at 1:20pm
4690K System
(12 items)
 
HTPC
(9 items)
 
HP dv6 laptop
(13 items)
 
CPUMotherboardGraphicsRAM
Core i5 4690K ASRock Z97 Extreme4 XFX Radeon 7950 32 GB DDR3-2133 
Hard DriveOptical DriveCoolingOS
Samsung 850 EVO SSD Samsung DVD/CD-writer Corsair Hydro H60 Windows 7 
MonitorMonitorPowerCase
ASUS PA248Q Dell U2412M XFX 850W Black Edition XXX Fractal Design Arc Midi 2 
CPUMotherboardGraphicsRAM
Pentium Dual Core E6700 MSI G41M-P33 Combo ATI HD4350 Kingston ValueRAM DDR3-1333 
RAMHard DriveOSMonitor
Kingston ValueRAM DDR3-1333 WD Caviar Blue Windows 7 64-bit Sony 32" TV set 
Case
Apex TX-381 
CPUGraphicsRAMHard Drive
Core i5 430M GT230M (1 Gb dedicated) 8 GB DDR3-1066 640 GB 
OSMonitor
Windows 7 Home Premium (64-bit) 15.6" 
  hide details  
4690K System
(12 items)
 
HTPC
(9 items)
 
HP dv6 laptop
(13 items)
 
CPUMotherboardGraphicsRAM
Core i5 4690K ASRock Z97 Extreme4 XFX Radeon 7950 32 GB DDR3-2133 
Hard DriveOptical DriveCoolingOS
Samsung 850 EVO SSD Samsung DVD/CD-writer Corsair Hydro H60 Windows 7 
MonitorMonitorPowerCase
ASUS PA248Q Dell U2412M XFX 850W Black Edition XXX Fractal Design Arc Midi 2 
CPUMotherboardGraphicsRAM
Pentium Dual Core E6700 MSI G41M-P33 Combo ATI HD4350 Kingston ValueRAM DDR3-1333 
RAMHard DriveOSMonitor
Kingston ValueRAM DDR3-1333 WD Caviar Blue Windows 7 64-bit Sony 32" TV set 
Case
Apex TX-381 
CPUGraphicsRAMHard Drive
Core i5 430M GT230M (1 Gb dedicated) 8 GB DDR3-1066 640 GB 
OSMonitor
Windows 7 Home Premium (64-bit) 15.6" 
  hide details  
post #2 of 3
Try reading this: http://floating-point-gui.de/errors/comparison/
Webcrawler
(17 items)
 
  
CPUMotherboardGraphicsRAM
i5 3570k ASRock Z75 Pro3 Sapphire 7870 XT Boost Corsair Vengeance, DDR3 1600Mhz 
Hard DriveHard DriveOSMonitor
SpinPoint F1 1TB 64GB M4 SSD Windows 8.1 SyncMaster P2050 
MonitorKeyboardPowerMouse
Dell U2312HM Sidewinder X4 Be Quiet! Pure Power CM L8 430w Zowie FK 
AudioAudio
Xonar DG Sennheiser HD 555 
  hide details  
Webcrawler
(17 items)
 
  
CPUMotherboardGraphicsRAM
i5 3570k ASRock Z75 Pro3 Sapphire 7870 XT Boost Corsair Vengeance, DDR3 1600Mhz 
Hard DriveHard DriveOSMonitor
SpinPoint F1 1TB 64GB M4 SSD Windows 8.1 SyncMaster P2050 
MonitorKeyboardPowerMouse
Dell U2312HM Sidewinder X4 Be Quiet! Pure Power CM L8 430w Zowie FK 
AudioAudio
Xonar DG Sennheiser HD 555 
  hide details  
post #3 of 3
Thread Starter 
Okay, thanks. smile.gif Looks like an epsilon will work smile.gif
4690K System
(12 items)
 
HTPC
(9 items)
 
HP dv6 laptop
(13 items)
 
CPUMotherboardGraphicsRAM
Core i5 4690K ASRock Z97 Extreme4 XFX Radeon 7950 32 GB DDR3-2133 
Hard DriveOptical DriveCoolingOS
Samsung 850 EVO SSD Samsung DVD/CD-writer Corsair Hydro H60 Windows 7 
MonitorMonitorPowerCase
ASUS PA248Q Dell U2412M XFX 850W Black Edition XXX Fractal Design Arc Midi 2 
CPUMotherboardGraphicsRAM
Pentium Dual Core E6700 MSI G41M-P33 Combo ATI HD4350 Kingston ValueRAM DDR3-1333 
RAMHard DriveOSMonitor
Kingston ValueRAM DDR3-1333 WD Caviar Blue Windows 7 64-bit Sony 32" TV set 
Case
Apex TX-381 
CPUGraphicsRAMHard Drive
Core i5 430M GT230M (1 Gb dedicated) 8 GB DDR3-1066 640 GB 
OSMonitor
Windows 7 Home Premium (64-bit) 15.6" 
  hide details  
4690K System
(12 items)
 
HTPC
(9 items)
 
HP dv6 laptop
(13 items)
 
CPUMotherboardGraphicsRAM
Core i5 4690K ASRock Z97 Extreme4 XFX Radeon 7950 32 GB DDR3-2133 
Hard DriveOptical DriveCoolingOS
Samsung 850 EVO SSD Samsung DVD/CD-writer Corsair Hydro H60 Windows 7 
MonitorMonitorPowerCase
ASUS PA248Q Dell U2412M XFX 850W Black Edition XXX Fractal Design Arc Midi 2 
CPUMotherboardGraphicsRAM
Pentium Dual Core E6700 MSI G41M-P33 Combo ATI HD4350 Kingston ValueRAM DDR3-1333 
RAMHard DriveOSMonitor
Kingston ValueRAM DDR3-1333 WD Caviar Blue Windows 7 64-bit Sony 32" TV set 
Case
Apex TX-381 
CPUGraphicsRAMHard Drive
Core i5 430M GT230M (1 Gb dedicated) 8 GB DDR3-1066 640 GB 
OSMonitor
Windows 7 Home Premium (64-bit) 15.6" 
  hide details  
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Coding and Programming
This thread is locked  
Overclock.net › Forums › Software, Programming and Coding › Coding and Programming › Floating Point Comparisons in VB.NET