I think you are talking about exploits.
there are two general kinds of exploits local and remote. Remote exploits use the internet to send a payload to a certain service to overflow it and execute code.. While a local service will do the same but not remotely.. simple really..
A buffer overflow works as said above by writing more data then the buffer allowed.. hence "buffer" overflow an example in c is.
#include
int main(int argc,char *argv[]){
char *buff[20];
strcpy(buff,argv[1]);
}
Now when you run the program
"c:\lala.exe AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA"
more data is written to the buffer then was allocated and the registers get overwritten if your in windows a error box will pop up and you can find out which registers you overwrote by looking for 41 which is the hex value of "A"
it should overwrite the esp and ebp if you have enuf data seeing as how the buffer is 20 bytes and you wrote more than 21 bytes u overwrite the register why did i say 21 well thats because of the null terminated byte every buffer has '\0' (thats a basic explanation of it all) How do exploits work.. they do just that they overflow buffers and execute shellcode (ahh what is shellcode?) shellcode is opcode(operation code) of an asm program an example .
#include <windows.h>
#include <winbase.h>
void main()
{
LoadLibrary("msvcrt.dll");
__asm {
mov esp,ebp
push ebp
mov ebp,esp
xor edi,edi
push edi
sub esp,04h
mov byte ptr [ebp-08h],63h
mov byte ptr [ebp-07h],6Dh
mov byte ptr [ebp-06h],64h
mov byte ptr [ebp-05h],2Eh
mov byte ptr [ebp-04h],65h
mov byte ptr [ebp-03h],78h
mov byte ptr [ebp-02h],65h
mov eax, 0x77c28044 //put your system() address here
push eax
lea eax,[ebp-08h]
push eax
call dword ptr [ebp-0ch]
}
}
this program just runs cmd.exe using the system function and when u convert this to opcode you have your shellcode.. to get the system address you just use a debugger to find out where the function is stored in memory
Ok thats shellcode.. you understand buffer overflows well the simple version.. So now we make an exploit what this does is overwrite the return address of the program so when it tries to return it executes or shellcode..
there is no easy way to explain this..
but basically you store your shellcode in public memory range 0x00000000 to 0x7FFFFFFF
and overwrite the stack with your shellcodes memory address and execute it. Voila
just google for buffer overflow tutorials
for a more in depth idea of what they are