Files
NoteNextra-origin/content/CSE4303/CSE4303_L16.md
Zheyuan Wu 04cda8c4ca updates
2026-03-24 15:28:40 -05:00

6.2 KiB

CSE4303 Introduction to Computer Security (Lecture 16)

System security

  • Why system security / platform security?
    • All code runs on some physical machine!
      • The cloud is not a cloud
      • Web pages are just data and code copied from a server that also manages the transfer
  • Why Linux?
    • Majority of web servers run Linux (esp. Cloud); popular in embedded, mobile devices

Operating system background

Context: computing stack

Layer Description
Application Web browser, user apps, DNS
OS:libs Memory allocations, compiler/linker
OS:kernel Process control, networking, file system, access control
OS:drivers Manage hardware
(Firmware) Minimal hardware management (if no full OS)
Hardware Processor, cahce, RAM, disk, USB ports

Operating systems

  • Operating System:

    • Provides easier to use and high level abstractions for resources such as address space for memory and files for disk blocks.
    • Provides controlled access to hardware resources.
    • Provides isolation between different processes and between the processes running untrusted/application code and the trusted operating system.
  • Need for trusting an operating system

    • Why do we need to trust the operating system? (AKA a Trusted Computing Base or TCB)
    • What requirements must it meet to be trusted?
  • TCB Requirements:

      1. Tamper-proof
      1. Complete mediation (reference monitor)
      1. Correct

Isolating OS from Untrusted User Code

  • How do we meet the first requirement of a TCB (e.g., isolation or tamper-proofness)?
    • Hardware support for memory protection
    • Processor execution modes (system AND user modes, execution rings)
    • Privileged instructions which can only be executed in system mode
    • System calls used to transfer control between user and system code

System Calls: Going from User to OS Code

  • System calls used to transfer control between user and system code
    • Such calls come through "call gates" and return back to user code.
      • The processor execution mode or privilege ring changes when call and return happen.
    • x86 sysenter / sysexit instructions

Isolating User Processes from Each Other

  • How do we meet the user/user isolation and separation?
    • OS uses hardware support for memory protection to ensure this.

Virtualization

  • OS is large and complex, even different operating systems may be desired by different customers
  • Compromise of an OS impacts all applications

Complete Mediation: The TCB

  • Make sure that no protected resource (e.g., memory page or file) could be accessed without going through the TCB
  • TCB acts as a reference monitor that cannot be bypassed
  • Privileged instructions

Limiting the Damage oa a Hacked OS

Use: Hypervisor, virtual machines, guest OS and applications

Compromise of OS in VM1 only impacts applications running on VM1

Secure boot and Root of Trust (RoT)

Goal: create chain of trust back to hardware-stored cryptographic keys

Secure enclave: overview (Intel SGX)

Intel SGX

Goal: keep sensitive data within hardware-isolated encrypted environment

Access control

Controlling Accesses to Resources

  • TCB (reference monitor) sees a request for a resource, how does it decide whether it should be granted?

    • Example: Should John's process making a request to read a certain file be allowed to do so?
  • Authentication establishes the source of a request (e.g., John's UID)

  • Authorization (or access control) answers the question if a certain source of a request (User ID) is allowed to read the file

  • Subject who owns a resource (creates it) should be able to control access to it (sometimes this is not true)

  • Access control

    • Basically, it is about who is allowed to access what.
    • Two parts
      • Part I - Policy: decide who should have access to certain resources (access control policy)
      • Part II - Enforcement: only accesses defined by the access control policy are granted.
    • Complete mediation is essential for successful enforcement

Discretionary Access Control

  • In discretionary access control (DAC), owner of a resource decides how it can be shared
    • Owner can choose to give read or write access to other users
  • Two problems with DAC:
    • You cannot control if someone you share a file with will not further share the data contained in it
      • Cannot control "information flow"
    • In many organizations, a user does not get to decide how certain type of data can be shared
      • Typically the employer may mandate how to share various types of sensitive data
    • Mandatory Access Control (MAC) helps address these problems

Mandatory Access Control (MAC) Models

  • User works in a company and the company decides how data should be shared
    • Hospital owns patient records and limits their sharing
      • Regulatory requirements may limit sharing
      • HIPAA for health information

Example: Linux system controls

Unix file access control list

  • Each file has owner and group
  • Permissions set by owner
    • Read, write, execute
    • Owner, group, other
    • Represented by vector of four octal values
  • Only owner, root can change permissions
    • This privilege cannot be delegated or shared
  • Setid bits -- Discuss in a few slides

Process effective user id (EUID)

  • Each process has three IDs (+ more under Linux)
    • Real user ID (RUID)
      • Same as the user ID of parent (unless changed)
      • Used to determine which user started the process
    • Effective user ID (EUID)
      • From set user ID bit on the file being executed, or sys call
      • Determines the permissions for process
        • File access and port binding
    • Saved user ID (SUID)
      • So previous EUID can be restored
  • Real group ID, effective group ID used similarly

Weaknesses in Unix isolation, privileges

  • Shared resources
    • Since any process can create files in /tmp directory, an untrusted process may create files that are used by arbitrary system processes
  • Time-of-Check-to-Time-of-Use (TOCTTOU), i.e. race conditions
    • Typically, a root process uses system call to determine if initiating user has permission to a particular file, e.g. /tmp/X.
    • After access is authorized and before the file open, user may change the file /tmp/X to a symbolic link to a target file /etc/shadow.

Hazard: race conditions