Style bullet-list with arrows

just add content in before and display inline-block

#arrow {
    border-right:2px solid black;
    border-bottom:2px solid black;
    width:10px;
    height:10px;
    transform: rotate(-45deg);
    margin-top:40px;
}

ul li {
    padding-bottom: 10px;
    list-style:none;
}

ul li:before {
    border-right: 2px solid black;
    border-bottom: 2px solid black;
    width: 10px;
    height: 10px;
    transform: rotate(-45deg);
    content: "";
    display: inline-block;
    margin-right: 5px;
}

Not directly answering the question asked, but hopefully useful to some people who found this question from a search like I did!

Using roughly the same idea as other answers, but with a simpler ::before pseudo-element, you can use any unicode arrow character for your bullet rather than messing about with borders on divs:

ul {
  position: relative;
  list-style: none;
}

li::before {
  content: '▶';
  position: absolute;
  left: 0;
}

Here is a list of unicode arrows, so you can find something that you like: http://xahlee.info/comp/unicode_arrows.html

Use content: '' with pseudo elements (:before or :after). And use list-style: none for ul to remove the bullets. Like:

ul {
  list-style: none;
}

ul li:before{
   content: '';
   position: absolute;
   border-right:2px solid black;
   border-bottom:2px solid black;
   width:10px;
   height:10px;
   top: calc(50% - 4px);
   left: -20px;
   transform: translateY(-50%) rotate(-45deg);
}

Have a look at the snippet below:

#arrow {
    border-right:2px solid black;
    border-bottom:2px solid black;
    width:10px;
    height:10px;
    transform: rotate(-45deg);
    margin-top:40px;
}



ul li {
  position: relative;
	padding-bottom: 10px;
}

ul {
  list-style: none;
}

ul li:before{
   content: '';
   position: absolute;
   border-right:2px solid black;
   border-bottom:2px solid black;
   width:10px;
   height:10px;
   top: calc(50% - 4px);
   left: -20px;
   transform: translateY(-50%) rotate(-45deg);
}

<!-- Want to place arrow where bullet points are  -->
<ul>
  <li>Item #1</li>
  <li>Item #2</li>
  <li>Item #3</li>
  <li>Item #4</li>
  <li>Item #5</li>
</ul>

Hope this helps!

A more modern (2021) approach would be:

Note: The centering and the colors are optional.

ul {
  list-style-type: none
}

li {
  display: grid;
  grid-template-columns: 20px auto;
  justify-content: start;
  align-items: center;
}

li::before {
  content: ">";
  font-size: 10px;
  color: #999;
}

<ul>
<li>Item 1</li>
<li>Item 2</li>
</ul>