16進の割り算がおかしい
Posted: 2007年12月21日(金) 12:24
AB Ver5 CP4(Rev.265)を使用
コメント部はABが計算した結果です。
Dim a As QWord
a = &h1234567812345678 / &h10 :' a = &h123456781234567(正常)
Dim a1 As QWord
Dim a2 As QWord
a1 = &h1234567812345678
a2 = (a1 / &h10) As QWord :' a2 = &h123456800000000(誤)
Dim b1 As DWord
Dim b2 As DWord
b1 = &h12345678
b2 = (b1 / &h10) As DWord :' a2 = &h1234568(誤:四捨五入?)
Dim c1 As Word
Dim c2 As Word
c1 = &h5678
c2 = (c1 / &h10) As Word :' a2 = &h567(正常)
コメント部はABが計算した結果です。
Dim a As QWord
a = &h1234567812345678 / &h10 :' a = &h123456781234567(正常)
Dim a1 As QWord
Dim a2 As QWord
a1 = &h1234567812345678
a2 = (a1 / &h10) As QWord :' a2 = &h123456800000000(誤)
Dim b1 As DWord
Dim b2 As DWord
b1 = &h12345678
b2 = (b1 / &h10) As DWord :' a2 = &h1234568(誤:四捨五入?)
Dim c1 As Word
Dim c2 As Word
c1 = &h5678
c2 = (c1 / &h10) As Word :' a2 = &h567(正常)