In this work, we explore the use of Large Language Models (LLMs) for the challenging task of long-range dialog modeling. While LLMs have excelled in various Natural Language Processing (NLP) tasks, adapting them for extended dialog contexts poses challenges due to computational overhead and data requirements. LLMs often struggle with fixed context window sizes, limiting their application in lengthy conversations. In this work, we leverage LLMs' contextual learning capabilities using instruction prompts and retrieval-based context augmentation, without any fine-tuning. We focus on long-term dialog modeling, addressing challenges like data independence, avoiding fine-tuning, and accommodating the context of long conversations within shorter windows. Our empirical experiments on two datasets, namely Multi-Session Chat and MultiDoc2Dial demonstrate how including relevant information in LLMs' input context affects dialog generation performance while reducing computational costs associated with longer contexts.