How can I add extra kerning around -- (en-dash) in XeTeX

I don't think that it is possible intercharclass, as the ligature from -- to endash is built later. What you can do is add a thin space through a mapping.

E.g. with this mapping you get a small space between 1 and the dashes:

; TECkit mapping for TeX input conventions <-> Unicode characters

LHSName "kerndash"
RHSName "UNICODE"

pass(Unicode)

; ligatures from Knuth's original CMR fonts
U+002D U+002D                   <>      U+2013  ; -- -> en dash
U+002D U+002D U+002D    <>      U+2014  ; --- -> em dash

pass(Unicode)
U+0031 U+2013 <>  U+0031 U+2009 U+2013
U+0031 U+2014 <>  U+0031 U+2009 U+2014

This answer shows how to compile it Change em-dash character in an OpenType font with xetex

Test document:

\documentclass{article}
\usepackage{fontspec}
\setmainfont{Arial}
\setsansfont[Mapping=kerndash]{Arial}
\begin{document}
1– 1-- 1— 1---

{\sffamily 1– 1-- 1— 1---}

\end{document}

(The OP has asked primarily for a XeLaTeX-based approach. However, he has also encouraged the submission of a LuaLaTeX-based solution.)

Here's a LuaLaTeX-based solution. It works with en- and em-dashes encoded either as -- and --- or as – and —, respectively. LaTeX macros are provided to switch the operation of the Lua function that performs the insertions of the extra kerns (at the stage of the callback process_input_buffer, i.e., very early during the processing phase) on and off, as needed. The amount of extra kerning that inserted between the numbers and the en/em-dashes can be modified via a LaTeX macro called \kset, e.g., \kset{0.07em}. (Restore the default amount of kerning by executing \kset{0.07em}.)

% !TEX TS-program = lualatex
\documentclass{article}
\usepackage{fontspec}

\usepackage{luacode}
\begin{luacode}
kval = "0.05em" -- Set default value of amount of extra kerning
function kern_dashes ( s )
  s = string.gsub ( s , "(%d)%-%-" , "%1\\kern"..kval.."--" )
  s = string.gsub ( s , "%-%-(%d)" , "--\\kern"..kval.."%1" ) 
  s = unicode.utf8.gsub ( s, "(%d)–" , "%1\\kern"..kval.."–" )
  s = unicode.utf8.gsub ( s, "–(%d)" , "–\\kern"..kval.."%1" ) 
  s = unicode.utf8.gsub ( s, "(%d)—" , "%1\\kern"..kval.."—" )
  s = unicode.utf8.gsub ( s, "—(%d)" , "—\\kern"..kval.."%1" ) 
  return s 
end
\end{luacode}
\newcommand{\extrakernOn}{\directlua{
    luatexbase.add_to_callback (   
    "process_input_buffer" , kern_dashes , "kern_dashes" )}}
\newcommand{\extrakernOff}{\directlua{
    luatexbase.remove_from_callback (   
    "process_input_buffer" , "kern_dashes" )}}
\AtBeginDocument{\extrakernOn} % switch function on by default

% Macro to set the amount of extra kerning
\newcommand\kset[1]{\directlua{kval=\luastring{#1}}} 

\begin{document}
\obeylines % just for this example

3--5, 3---5
3–5, 3—5

\extrakernOff % switch off automatic insertion of extra kerns
3--5, 3---5
3–5, 3—5

\extrakernOn% switch automatic insertion of extra kerns back on
\kset{.2em} % quadruple the amount of extra kerning
3--5, 3---5
3–5, 3—5

\end{document}

---

Addendum: In the preceding example, the Lua function kern_dashes is written so that extra kerns are inserted before the start of en- and em-dashes (assuming they're immediately preceded by a digit) or after the end of en- and em-dashes (assuming they're immediately followed by a digit). If the kerns should be inserted only if the en- and em-dashes are both preceded and followed by digits, the following code would be more appropriate:

function kern_dashes ( s )
  s = string.gsub ( s , "(%d)%-%-(%d)"   ,  "%1\\kern"..kval.."--\\kern"..kval.."%2" )
  s = string.gsub ( s , "(%d)%-%-%-(%d)" ,  "%1\\kern"..kval.."---\\kern"..kval.."%2" )
  s = unicode.utf8.gsub ( s , "(%d)–(%d)" , "%1\\kern"..kval.."–\\kern"..kval.."%2" )
  s = unicode.utf8.gsub ( s , "(%d)—(%d)" , "%1\\kern"..kval.."—\\kern"..kval.."%2" )
  return s 
end