3rd party sign in broken because of secure cookie

Sign in with GitHub/Discord (and probably Twitter) not working on my configuration (Misskey http <-> apache proxy https <-> client https)

Error: Cannot send secure cookie over unencrypted connection when trying to access /api/signin/... (client: Internal Server Error)