Find the closest three-digit hex colour

05AB1E, 13 bytes

ćs2ôH8+17÷hJ«

Try it online!

How?

ćs2ôH8+17÷hJ« | string, S   e.g. stack: "#B23F08"
ć             | decapitate              "B23F08", "#"
 s            | swap                    "#", "B23F08"
  2           | two                     "#", "B23F08", 2
   ô          | chuncks                 "#", ["B2", "3F", "08"]
    H         | from hexadecimal        "#", [178, 63, 8]
     8        | eight                   "#", [178, 63, 8], 8
      +       | add                     "#", [186, 71, 16]
       17     | seventeen               "#", [186, 71, 16], 17
         ÷    | integer divide          "#", [10, 4, 0]
          h   | to hexadecimal          "#", ["A", "4", "0"]
           J  | join                    "#", "A40"
            « | concatenate             "#A40"
              | print top of stack

Japt, 16 bytes

r"%w"²_n16_r17Ãg

Try it or run all test cases

r"%w"²_n16_r17Ãg     :Implicit input of string
r                    :Replace
 "%w"                :RegEx /\w/g
     ²               :Duplicate, giving /\w\w/g
      _              :Pass each match through a function
       n16           :  Convert to decimal
          _          :  Pass through the following function, and convert back to hex
           r17       :    Round to the nearest multiple of 17
              Ã      :  End function
               g     :  Get first character

8088 Assembly, IBM PC DOS, 59 58 bytes

Unassembled listing:

BE 0082     MOV  SI, 82H    ; SI to begining of input string 
AC          LODSB           ; load first '#' char into AL 
B4 0E       MOV  AH, 0EH    ; BIOS display char function  
CD 10       INT  10H        ; call BIOS 
B3 11       MOV  BL, 17     ; set up for divide by 17 
B9 0304     MOV  CX, 0304H  ; hex byte loop counter (CH=3), shift counter (CL=4) 
        LOOP_BYTE: 
AD          LODSW           ; load next two ASCII hex chars into AX 
B7 02       MOV  BH, 2      ; hex chars loop counter
        LOOP_ALPHA:
2C 30       SUB  AL, '0'    ; convert from ASCII 
3C 0A       CMP  AL, 10     ; is digit > 10 (A-F)? 
7C 02       JL   NOT_ALPHA  ; if not, jump to next char
2C 07       SUB  AL, 7      ; ASCII adjust alpha char to binary 
        NOT_ALPHA: 
86 E0       XCHG AH, AL     ; swap first and second chars 
FE CF       DEC  BH         ; decrement loop counter
75 F2       JNZ  LOOP_ALPHA ; loop to next hex char
D2 E0       SHL  AL, CL     ; shift low nibble to high nibble 
02 C4       ADD  AL, AH     ; add first and second nibbles
32 E4       XOR  AH, AH     ; clear AH for add/division
05 0008     ADD  AX, 8      ; add 0.5 (8/16) to round (with overflow) 
F6 F3       DIV  BL         ; divide by 17 
3C 0A       CMP  AL, 10     ; is digit > 10? 
7C 02       JL   DISP_CHAR  ; if not, jump to display digit 
04 07       ADD  AL, 7      ; binary adjust alpha char to ASCII 
        DISP_CHAR: 
04 30       ADD  AL, '0'    ; convert to ASCII 
B4 0E       MOV  AH, 0EH    ; BIOS display char function  
CD 10       INT  10H        ; call BIOS 
FE CD       DEC  CH         ; decrement loop counter 
75 D4       JNZ  LOOP_BYTE  ; loop to next hex byte
C3          RET             ; return to DOS 

Standalone PC DOS executable. Input is via command line, output is to console.

Most of the code length is handling the conversion of required hex string I/O into bytes, since DOS/x86 machine code has no built-ins for that.

I/O:

Download and test HEXCLR.COM, or xxd hexdump:

0000000: be82 00ac b40e cd10 b311 b904 03ad b702  ................
0000010: 2c30 3c0a 7c02 2c07 86e0 fecf 75f2 d2e0  ,0<.|.,.....u...
0000020: 02c4 32e4 0508 00f6 f33c 0a7c 0204 0704  ..2......<.|....
0000030: 30b4 0ecd 10fe cd75 d4c3                 0......u..