Overclock.net › Forums › Software, Programming and Coding › Coding and Programming › Data Payload Decoding - Issues with Byte Reading
New Posts  All Forums:Forum Nav:

Data Payload Decoding - Issues with Byte Reading - Page 2

post #11 of 30
Weird! I always assumed one char was one 8 bit byte, myself. Is there any way you can just handle the stream in binary mode?
4690K System
(12 items)
 
HTPC
(9 items)
 
HP dv6 laptop
(13 items)
 
CPUMotherboardGraphicsRAM
Core i5 4690K ASRock Z97 Extreme4 XFX Radeon 7950 32 GB DDR3-2133 
Hard DriveOptical DriveCoolingOS
Samsung 850 EVO SSD Samsung DVD/CD-writer Corsair Hydro H60 Windows 7 
MonitorMonitorPowerCase
ASUS PA248Q Dell U2412M XFX 850W Black Edition XXX Fractal Design Arc Midi 2 
CPUMotherboardGraphicsRAM
Pentium Dual Core E6700 MSI G41M-P33 Combo ATI HD4350 Kingston ValueRAM DDR3-1333 
RAMHard DriveOSMonitor
Kingston ValueRAM DDR3-1333 WD Caviar Blue Windows 7 64-bit Sony 32" TV set 
Case
Apex TX-381 
CPUGraphicsRAMHard Drive
Core i5 430M GT230M (1 Gb dedicated) 8 GB DDR3-1066 640 GB 
OSMonitor
Windows 7 Home Premium (64-bit) 15.6" 
  hide details  
Reply
4690K System
(12 items)
 
HTPC
(9 items)
 
HP dv6 laptop
(13 items)
 
CPUMotherboardGraphicsRAM
Core i5 4690K ASRock Z97 Extreme4 XFX Radeon 7950 32 GB DDR3-2133 
Hard DriveOptical DriveCoolingOS
Samsung 850 EVO SSD Samsung DVD/CD-writer Corsair Hydro H60 Windows 7 
MonitorMonitorPowerCase
ASUS PA248Q Dell U2412M XFX 850W Black Edition XXX Fractal Design Arc Midi 2 
CPUMotherboardGraphicsRAM
Pentium Dual Core E6700 MSI G41M-P33 Combo ATI HD4350 Kingston ValueRAM DDR3-1333 
RAMHard DriveOSMonitor
Kingston ValueRAM DDR3-1333 WD Caviar Blue Windows 7 64-bit Sony 32" TV set 
Case
Apex TX-381 
CPUGraphicsRAMHard Drive
Core i5 430M GT230M (1 Gb dedicated) 8 GB DDR3-1066 640 GB 
OSMonitor
Windows 7 Home Premium (64-bit) 15.6" 
  hide details  
Reply
post #12 of 30
Thread Starter 
Quote:
Originally Posted by Quantum Reality View Post

Weird! I always assumed one char was one 8 bit byte, myself. Is there any way you can just handle the stream in binary mode?

I thought it was 8-bits also. I don't think I will be able to; however, I could mask the first byte and extract the second byte's data. I assume it'll be zero. Hang on a min.
post #13 of 30
Thread Starter 
Hmm odd.

Whenever the most significant bit is set (to signal there is more data to come (so >127)) the second byte is on (255/0xFF). Slightly confused.

I must be treating the data incorrectly.
Edited by mikeo01 - 4/20/14 at 11:27am
post #14 of 30
I am at a bar right now, I am drinking my sorrows away. I will help out when I can. ^.^
Snowdevil
(16 items)
 
ASUS G750JM
(9 items)
 
 
CPUMotherboardGraphicsGraphics
[i7 4790K @ 4.4 GHz (1.186v)] [Asus Sabertooth Z97 Mark S] [nVidia Geforce GTX 1080] [nVidia Geforce GTX 1080] 
RAMHard DriveCoolingOS
[G.Skill 32GB DDR3 2133 MHz] [Crucial MX100 256GB] [Phanteks PH-TC12DX] [Win 10.1 Pro] 
MonitorMonitorKeyboardPower
[LG 29UM65 (2560x1080)] [QNIX Evo II LED (2560x1440)] [WASD v2 Tenkeyless] [NZXT Hale90 v2 ] 
CaseMouseMouse PadAudio
[ThermalTake GT10 Snow Edition] [Razer Mamba - Chroma] [Razer Kabuto] [Razer Man O' War] 
CPUMotherboardGraphicsRAM
i7 4770HQ Intel HM87 Express Chipset Geforce GTX 860M 8GB DDR3L 1600 MHz 
Hard DriveOptical DriveCoolingOS
Samsung SSD EVO DVD-RW Stock Windows 8.1 
Monitor
1920x1080 TN 
  hide details  
Reply
Snowdevil
(16 items)
 
ASUS G750JM
(9 items)
 
 
CPUMotherboardGraphicsGraphics
[i7 4790K @ 4.4 GHz (1.186v)] [Asus Sabertooth Z97 Mark S] [nVidia Geforce GTX 1080] [nVidia Geforce GTX 1080] 
RAMHard DriveCoolingOS
[G.Skill 32GB DDR3 2133 MHz] [Crucial MX100 256GB] [Phanteks PH-TC12DX] [Win 10.1 Pro] 
MonitorMonitorKeyboardPower
[LG 29UM65 (2560x1080)] [QNIX Evo II LED (2560x1440)] [WASD v2 Tenkeyless] [NZXT Hale90 v2 ] 
CaseMouseMouse PadAudio
[ThermalTake GT10 Snow Edition] [Razer Mamba - Chroma] [Razer Kabuto] [Razer Man O' War] 
CPUMotherboardGraphicsRAM
i7 4770HQ Intel HM87 Express Chipset Geforce GTX 860M 8GB DDR3L 1600 MHz 
Hard DriveOptical DriveCoolingOS
Samsung SSD EVO DVD-RW Stock Windows 8.1 
Monitor
1920x1080 TN 
  hide details  
Reply
post #15 of 30
chars are 1 byte. That is defined in the C and C++ standard.

You should just stick with unsigned types in your code. use "unsigned char", or better yet, uint8_t from stdint.h, instead of char.
eyefinity
(22 items)
 
  
CPUMotherboardGraphicsGraphics
Intel Core i7 2600K EVGA E685 Radeon HD 6970 Radeon HD 6970 
RAMRAMHard DriveCooling
G.Skill SNIPER G.Skill SNIPER Crucial M4 128GB 6970 EK Water Block Acetal + Nickel 
CoolingCoolingCoolingOS
Swiftech MCR320-QP Radiator Swiftech MCP655 Pump EK Supreme HF CPU Block Windows 7 Ultimate 
MonitorMonitorMonitorKeyboard
Dell E228WFP Dell E228WFP Dell P2210 G510 
PowerCaseMouseMouse Pad
TX850 HAF932 BE G500 XTRAC PADS Ripper 
Audio
G930 
  hide details  
Reply
eyefinity
(22 items)
 
  
CPUMotherboardGraphicsGraphics
Intel Core i7 2600K EVGA E685 Radeon HD 6970 Radeon HD 6970 
RAMRAMHard DriveCooling
G.Skill SNIPER G.Skill SNIPER Crucial M4 128GB 6970 EK Water Block Acetal + Nickel 
CoolingCoolingCoolingOS
Swiftech MCR320-QP Radiator Swiftech MCP655 Pump EK Supreme HF CPU Block Windows 7 Ultimate 
MonitorMonitorMonitorKeyboard
Dell E228WFP Dell E228WFP Dell P2210 G510 
PowerCaseMouseMouse Pad
TX850 HAF932 BE G500 XTRAC PADS Ripper 
Audio
G930 
  hide details  
Reply
post #16 of 30
Quote:
Originally Posted by morethantoast View Post

chars are 1 byte. That is defined in the C and C++ standard.

You should just stick with unsigned types in your code. use "unsigned char", or better yet, uint8_t from stdint.h, instead of char.

That is true, it depends on the compiler/framework he is operating in. I am assuming .NET, but if it's standard C++ outside of Visual Studio, then yes, char by default is only one byte. While I love VS, I learned (started) programming in C, terminaled into a Unix super computer. There is a lot to be desired I feel in Java and C# for hard coded memory management. IDE's usually though allow implicit casting disablement. This means that implicit casts / side-effect have to be converted to explicit code. No auto-assumption. Usually a good idea for low level memory storage and recall.

The good news though is there are different char types.

wchar_t = more than enough space for all possible symbols.
char16_t = 16-bit
char32_t = 32-bit

Working with char16 or char32_t may fix all issues without recoding. May. If not, there is an error in the code.
Edited by RagingCain - 4/21/14 at 12:46pm
Snowdevil
(16 items)
 
ASUS G750JM
(9 items)
 
 
CPUMotherboardGraphicsGraphics
[i7 4790K @ 4.4 GHz (1.186v)] [Asus Sabertooth Z97 Mark S] [nVidia Geforce GTX 1080] [nVidia Geforce GTX 1080] 
RAMHard DriveCoolingOS
[G.Skill 32GB DDR3 2133 MHz] [Crucial MX100 256GB] [Phanteks PH-TC12DX] [Win 10.1 Pro] 
MonitorMonitorKeyboardPower
[LG 29UM65 (2560x1080)] [QNIX Evo II LED (2560x1440)] [WASD v2 Tenkeyless] [NZXT Hale90 v2 ] 
CaseMouseMouse PadAudio
[ThermalTake GT10 Snow Edition] [Razer Mamba - Chroma] [Razer Kabuto] [Razer Man O' War] 
CPUMotherboardGraphicsRAM
i7 4770HQ Intel HM87 Express Chipset Geforce GTX 860M 8GB DDR3L 1600 MHz 
Hard DriveOptical DriveCoolingOS
Samsung SSD EVO DVD-RW Stock Windows 8.1 
Monitor
1920x1080 TN 
  hide details  
Reply
Snowdevil
(16 items)
 
ASUS G750JM
(9 items)
 
 
CPUMotherboardGraphicsGraphics
[i7 4790K @ 4.4 GHz (1.186v)] [Asus Sabertooth Z97 Mark S] [nVidia Geforce GTX 1080] [nVidia Geforce GTX 1080] 
RAMHard DriveCoolingOS
[G.Skill 32GB DDR3 2133 MHz] [Crucial MX100 256GB] [Phanteks PH-TC12DX] [Win 10.1 Pro] 
MonitorMonitorKeyboardPower
[LG 29UM65 (2560x1080)] [QNIX Evo II LED (2560x1440)] [WASD v2 Tenkeyless] [NZXT Hale90 v2 ] 
CaseMouseMouse PadAudio
[ThermalTake GT10 Snow Edition] [Razer Mamba - Chroma] [Razer Kabuto] [Razer Man O' War] 
CPUMotherboardGraphicsRAM
i7 4770HQ Intel HM87 Express Chipset Geforce GTX 860M 8GB DDR3L 1600 MHz 
Hard DriveOptical DriveCoolingOS
Samsung SSD EVO DVD-RW Stock Windows 8.1 
Monitor
1920x1080 TN 
  hide details  
Reply
post #17 of 30
Thread Starter 
Quote:
Originally Posted by RagingCain View Post

That is true, it depends on the compiler/framework he is operating in. I am assuming .NET, but if it's standard C++ outside of Visual Studio, then yes, char by default is only one byte. While I love VS, I learning programming in C, terminalled into a Unix super computer. There is a lot to be desired I feel in Java and C# for hardcoding memory management. IDE's usually though allow implicit casting disablement. This means that implicit casts / side-effect have to be converted to explicit code. No auto-assumption. Usually a good idea for low level memory storage and recall.

The good news though is there are different char types.

wchar_t = more than enough space for all possible symbols.
char16_t = 16-bit
char32_t = 32-bit

Working with char16 or char32_t may fix all issues without recoding. May. If not, there is an error in the code.

I am using standard C++ smile.gif

So, I wonder what could be causing it?

I am working with basic encoding rules (BER), ASN.1.

I think I am getting myself even more confused now to be honest tongue.gif

Here is what I am working off: HERE.

That is very simplified but paves the way.

Quote:
Originally Posted by morethantoast View Post

chars are 1 byte. That is defined in the C and C++ standard.

You should just stick with unsigned types in your code. use "unsigned char", or better yet, uint8_t from stdint.h, instead of char.

Problem is I cannot use unsigned char type; if I define my buffer as unsigned char RECV_FROM complains.
Code:
char *BUFF = (char *)malloc(65536); // Allocate large memory block
.........................................................................................

                SRC=recvfrom(SNMP_ANALYSE,BUFF,65536,0,0,0);


Thanks for all the help thumb.gif
post #18 of 30
Quote:
Originally Posted by mikeo01 View Post

Quote:
Originally Posted by RagingCain View Post

That is true, it depends on the compiler/framework he is operating in. I am assuming .NET, but if it's standard C++ outside of Visual Studio, then yes, char by default is only one byte. While I love VS, I learning programming in C, terminalled into a Unix super computer. There is a lot to be desired I feel in Java and C# for hardcoding memory management. IDE's usually though allow implicit casting disablement. This means that implicit casts / side-effect have to be converted to explicit code. No auto-assumption. Usually a good idea for low level memory storage and recall.

The good news though is there are different char types.

wchar_t = more than enough space for all possible symbols.
char16_t = 16-bit
char32_t = 32-bit

Working with char16 or char32_t may fix all issues without recoding. May. If not, there is an error in the code.

I am using standard C++ smile.gif

So, I wonder what could be causing it?

I am working with basic encoding rules (BER), ASN.1.

I think I am getting myself even more confused now to be honest tongue.gif

Here is what I am working off: HERE.

That is very simplified but paves the way.

Quote:
Originally Posted by morethantoast View Post

chars are 1 byte. That is defined in the C and C++ standard.

You should just stick with unsigned types in your code. use "unsigned char", or better yet, uint8_t from stdint.h, instead of char.

Problem is I cannot use unsigned char type; if I define my buffer as unsigned char RECV_FROM complains.
Code:
char *BUFF = (char *)malloc(65536); // Allocate large memory block
.........................................................................................

                SRC=recvfrom(SNMP_ANALYSE,BUFF,65536,0,0,0);


Thanks for all the help thumb.gif

Malloc overflow, I am guessing the memory allocation isn't zeroed out. Since I know it's an overflow I just looked for malloc, buffer, overflow.

http://stackoverflow.com/questions/9901366/try-to-buffer-overflow-value-allocated-by-malloc

If you can't sort it out, I would suggest a new way of storing the data, read from buffer to an array, then work with the data in the array, not a buffer.
Snowdevil
(16 items)
 
ASUS G750JM
(9 items)
 
 
CPUMotherboardGraphicsGraphics
[i7 4790K @ 4.4 GHz (1.186v)] [Asus Sabertooth Z97 Mark S] [nVidia Geforce GTX 1080] [nVidia Geforce GTX 1080] 
RAMHard DriveCoolingOS
[G.Skill 32GB DDR3 2133 MHz] [Crucial MX100 256GB] [Phanteks PH-TC12DX] [Win 10.1 Pro] 
MonitorMonitorKeyboardPower
[LG 29UM65 (2560x1080)] [QNIX Evo II LED (2560x1440)] [WASD v2 Tenkeyless] [NZXT Hale90 v2 ] 
CaseMouseMouse PadAudio
[ThermalTake GT10 Snow Edition] [Razer Mamba - Chroma] [Razer Kabuto] [Razer Man O' War] 
CPUMotherboardGraphicsRAM
i7 4770HQ Intel HM87 Express Chipset Geforce GTX 860M 8GB DDR3L 1600 MHz 
Hard DriveOptical DriveCoolingOS
Samsung SSD EVO DVD-RW Stock Windows 8.1 
Monitor
1920x1080 TN 
  hide details  
Reply
Snowdevil
(16 items)
 
ASUS G750JM
(9 items)
 
 
CPUMotherboardGraphicsGraphics
[i7 4790K @ 4.4 GHz (1.186v)] [Asus Sabertooth Z97 Mark S] [nVidia Geforce GTX 1080] [nVidia Geforce GTX 1080] 
RAMHard DriveCoolingOS
[G.Skill 32GB DDR3 2133 MHz] [Crucial MX100 256GB] [Phanteks PH-TC12DX] [Win 10.1 Pro] 
MonitorMonitorKeyboardPower
[LG 29UM65 (2560x1080)] [QNIX Evo II LED (2560x1440)] [WASD v2 Tenkeyless] [NZXT Hale90 v2 ] 
CaseMouseMouse PadAudio
[ThermalTake GT10 Snow Edition] [Razer Mamba - Chroma] [Razer Kabuto] [Razer Man O' War] 
CPUMotherboardGraphicsRAM
i7 4770HQ Intel HM87 Express Chipset Geforce GTX 860M 8GB DDR3L 1600 MHz 
Hard DriveOptical DriveCoolingOS
Samsung SSD EVO DVD-RW Stock Windows 8.1 
Monitor
1920x1080 TN 
  hide details  
Reply
post #19 of 30
Thread Starter 
Quote:
Originally Posted by RagingCain View Post

Malloc overflow, I am guessing the memory allocation isn't zeroed out. Since I know it's an overflow I just looked for malloc, buffer, overflow.

http://stackoverflow.com/questions/9901366/try-to-buffer-overflow-value-allocated-by-malloc

If you can't sort it out, I would suggest a new way of storing the data, read from buffer to an array, then work with the data in the array, not a buffer.

It is in an array.

I am inputting data from the buffer straight into a standard char array (8-bit default?). Then looping over whilst ensuring the buffer is then freed.

The char array is passed over to a separate function to began processing on it, and a large memory block is set to ensure we don't run out of local memory.

I am also logging any errors that may occur on receiving datagrams using WSAGetLastError(), so any buffer overflows will be written to a log file.

Also, an overflow in a char array will cause the program to crash anyway shouldn't it?


Below is my code. STOP variable isn't used as of yet, it was going to be used later on but decided not to, so keeping it there for a loop so the buffer gets freed regardless.
Code:
void START_ANALYSIS(SOCKET SNMP_ANALYSE)
{
        char *BUFF = (char *)malloc(65536); // Allocate large memory block
        int SRC, STOP=0;

        if (BUFF==NULL) // Ensure BUFF isn't empty (means memory allocation failure
        {
        std::cout << "Memory Block Allocation - Failed.\n";
        return;
        }

        do // Loop around capturing packets IF
        {
                SRC=recvfrom(SNMP_ANALYSE,BUFF,65536,0,0,0); // Recieves all datagrams on this socket using large buffer (as to not buffer overload)

                if(SRC>0) // If datagram has been captured the buffer should be above 0 bytes
                {
                PROCESS(BUFF, SRC); // Send over what the socket has captured (raw data)
                }
                else // Otherwise do nothing with it
                {
                        ERROR_LOG << WSAGetLastError() << "\n";
                }
        }
        while (STOP==0); // Free buffer

free(BUFF);
}

Variable SRC is used for the size of the data.
post #20 of 30
calloc might solve the problem, since it apparently does some error checking malloc doesn't do.
4690K System
(12 items)
 
HTPC
(9 items)
 
HP dv6 laptop
(13 items)
 
CPUMotherboardGraphicsRAM
Core i5 4690K ASRock Z97 Extreme4 XFX Radeon 7950 32 GB DDR3-2133 
Hard DriveOptical DriveCoolingOS
Samsung 850 EVO SSD Samsung DVD/CD-writer Corsair Hydro H60 Windows 7 
MonitorMonitorPowerCase
ASUS PA248Q Dell U2412M XFX 850W Black Edition XXX Fractal Design Arc Midi 2 
CPUMotherboardGraphicsRAM
Pentium Dual Core E6700 MSI G41M-P33 Combo ATI HD4350 Kingston ValueRAM DDR3-1333 
RAMHard DriveOSMonitor
Kingston ValueRAM DDR3-1333 WD Caviar Blue Windows 7 64-bit Sony 32" TV set 
Case
Apex TX-381 
CPUGraphicsRAMHard Drive
Core i5 430M GT230M (1 Gb dedicated) 8 GB DDR3-1066 640 GB 
OSMonitor
Windows 7 Home Premium (64-bit) 15.6" 
  hide details  
Reply
4690K System
(12 items)
 
HTPC
(9 items)
 
HP dv6 laptop
(13 items)
 
CPUMotherboardGraphicsRAM
Core i5 4690K ASRock Z97 Extreme4 XFX Radeon 7950 32 GB DDR3-2133 
Hard DriveOptical DriveCoolingOS
Samsung 850 EVO SSD Samsung DVD/CD-writer Corsair Hydro H60 Windows 7 
MonitorMonitorPowerCase
ASUS PA248Q Dell U2412M XFX 850W Black Edition XXX Fractal Design Arc Midi 2 
CPUMotherboardGraphicsRAM
Pentium Dual Core E6700 MSI G41M-P33 Combo ATI HD4350 Kingston ValueRAM DDR3-1333 
RAMHard DriveOSMonitor
Kingston ValueRAM DDR3-1333 WD Caviar Blue Windows 7 64-bit Sony 32" TV set 
Case
Apex TX-381 
CPUGraphicsRAMHard Drive
Core i5 430M GT230M (1 Gb dedicated) 8 GB DDR3-1066 640 GB 
OSMonitor
Windows 7 Home Premium (64-bit) 15.6" 
  hide details  
Reply
New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Coding and Programming
Overclock.net › Forums › Software, Programming and Coding › Coding and Programming › Data Payload Decoding - Issues with Byte Reading