The video playlist feature was released just yesterday, but of course the MVP is quite limited.
I did however find our a neat trick: You can create a nice overview of the videos in the playlist using the embed webpart!
Simply use the embed webpart with the link of the playlist. Get the link to the playlist when you click View Playlist after adding a video to it.
Example link is: https://yourtenant.sharepoint.com/sites/jwtestgroup/Lists/Rabbits%20JW%20Test%20group/AllItems.aspx?env=WebViewList