Jan, this is a good request that i’ve noted with the product manager but although you can do this with XY plots it is not currently possible with the box plot.
With the box plot do you think you would want to label all points or just outliers?
in the mean time, annotations is probably your best work around, but realise this gets tiresome if you have a lot.