Protection through Segmentation

Checks are perform during logical to linear address translation when segmentation is enabled.

(1) Limit check uses the 20-bit limit field to ensure program does not access memory beyond the segment,  The processor also check the limit field in GDTR to ensure the segment selectors do not access entries beyond the GDT.

(2) Type check uses the S-flag and Type field to ensure the proper type is use.  For example, CS can only be loaded with code segment.  Access to the null descriptor will generate a general protection exception.

(3) Privilege check used privilege levels.  Current Privilege Level (CPL) is the RPL in the CS or SS register used by executing program.  CPL can be changed via a far call or jump instruction.  Privilege check happens when segment selector associated with segment descriptor is loaded.  This happens when program access data in another code segment or pass control to another segment.  Privilege violation generates a general protection exception.

To access data in another data segment, the selector must be loaded into the SS or one of the data segment (DS, ES, FS, GS).  To load selector into CS, it can only be done via instructions like JMP, CALL, RET, IRET, SYSENTER and SYSEXIT.

To access data in another segment, the DPL of target segment must be same or higher than CPL and RPL.

To load the stack segment register, both DPL of the stack segment and the corresponding RPL must be same as CPL.

When transferring control to a nonconforming code, the CPL must be equal to the DPL of destination segment.  In other words, the privilege level must be equal ob both sides of the fence.  In addition, the RPL of the selector for the destination segment must be less than or equal to the CPL.  Nonconforming code cannot be accessed by program with less privilege.

When transferring control to conforming code, the calling code's CPL must be greater than or equal to the DPL of the destination code.  RPL is not checked in this case.

(4) Restricted instruction check verify the program does not use privileged instruction like LGDT, LIDT, MOV a value to control register, HLT the processor, write to model specific register WRMSR etc.