Generative AI systems (ChatGPT, DALL-E, etc) are expanding into multiple areas of our lives, from art (Rombach et al. [2021]) to mental health (Rob Morris and Kareem Kouddous [2022]); their rapidly growing societal impact opens new opportunities, but also raises ethical concerns. The emerging field of AI alignment aims to make AI systems reflect human values. This paper focuses on evaluating the ethics of multimodal AI systems involving both text and images - a relatively under-explored area, as most alignment work is currently focused on language models. We first create a multimodal ethical database from human feedback on ethicality. Then, using this database, we develop algorithms, including a RoBERTa-large classifier and a multilayer perceptron, to automatically assess the ethicality of system responses.