Biography

Publications  

• Imagination Augmented Generation: Learning to Imagine Richer Context for Question Answering over Large Language Models   
  Huanxuan Liao , Shizhu He, Yao Xu, Yuanzhe Zhang, Kang Liu, Shengping Liu, Jun Zhao   
  Preprint , 2024.
  
   

   

   

   

   

   

   

   

   

  @misc{liao2024imagination,
  	title={Imagination Augmented Generation: Learning to Imagine Richer Context for Question Answering over Large Language Models}, 
  	author={Huanxuan Liao and Shizhu He and Yao Xu and Yuanzhe Zhang and Kang Liu and Shengping Liu and Jun Zhao},
  	year={2024},
  	eprint={2403.15268},
  	archivePrefix={arXiv},
  	primaryClass={cs.CL}
  }
  		

    Retrieval-Augmented-Generation and Gener-ation-Augmented-Generation have been proposed to enhance the knowledge required for
    question answering over Large Language Models (LLMs). However, the former depends on external resources, and both require incorporat-
    ing the explicit documents into the context, which results in longer contexts that lead to more resource consumption. Recent works indicate
    that LLMs have modeled rich knowledge, albeit not effectively triggered or activated. Inspired by this, we propose a novel knowledge-aug-
    mented framework, Imagination-Augmented-Generation (IAG), which simulates the human capacity to compensate for knowledge deficits 
    while answering questions solely through imagination, without relying on external resources. Guided by IAG, we propose an imagine richer 
    context method for question answering (IMcQA), which obtains richer context through the following two modules: explicit imagination by 
    generating a short dummy document with long context compress and implicit imagination with HyperNetwork for generating adapter weights. 
    Experimental results on three datasets demonstrate that IMcQA exhibits significant advantages in both open-domain and closed-book 
    settings, as well as in both in-distribution performance and out-of-distribution generalizations. Our code will be available at this https URL.
  		

• LMTuner: An user-friendly and highly-integrable Training Framework for fine-tuning Large Language Models   
  Yixuan Weng, Zhiqi Wang, Huanxuan Liao , Shizhu He, Shengping Liu, Kang Liu, Jun Zhao   
  Preprint , 2023.
  
   

   

   

   

   

   

   

   

   

   

  

    @misc{weng2023lmtuner,
  	title={LMTuner: An user-friendly and highly-integrable Training Framework for fine-tuning Large Language Models}, 
  	author={Yixuan Weng and Zhiqi Wang and Huanxuan Liao and Shizhu He and Shengping Liu and Kang Liu and Jun Zhao},
  	year={2023},
  	eprint={2308.10252},
  	archivePrefix={arXiv},
  	primaryClass={cs.CL}
    }
  		

    With the burgeoning development in the realm of large language models (LLMs), the demand for efficient incremental training tailored to
    specific industries and domains continues to increase. Currently, the predominantly employed frameworks lack modular design, it often 
    takes a lot of coding work to kickstart the training of LLM. To address this, we present "LMTuner", a highly usable, integrable, and scal-
    able system for training LLMs expeditiously and with minimal user-input. LMTuner comprises three main modules - the Interaction, Train-
    ing, and Inference Modules. We advocate that LMTuner’s usability and integrality alleviate the complexities in training large language 
    models. Remarkably, even a novice user could commence raining large language models within five minutes. Furthermore, it integrates
    DeepSpeed frameworks and supports Efficient Fine-Tuning methodologies like Low Rank Adaptation (LoRA),Quantized LoRA (QLoRA), etc., 
    enabling the training of language models scaling from 300M to a whopping 130B parameters using a single server.
  		



• Dynamic Weighted Neural Bellman-Ford Network for Knowledge Graph Reasoning   
  Huanxuan Liao , Shizhu He, Yao Xu, Kang Liu, Jun Zhao   
  Accepted to the 8th China Conference on Knowledge Graph and Semantic Computing (CCKS) , 2023.
  
   

   

   

   

   

    @InProceedings{10.1007/978-981-99-7224-1_1,
  	author="Liao, Huanxuan
  	and He, Shizhu
  	and Xu, Yao
  	and Liu, Kang
  	and Zhao, Jun",
  	editor="Wang, Haofen
  	and Han, Xianpei
  	and Liu, Ming
  	and Cheng, Gong
  	and Liu, Yongbin
  	and Zhang, Ningyu",
  	title="Dynamic Weighted Neural Bellman-Ford Network for Knowledge Graph Reasoning",
  	booktitle="Knowledge Graph and Semantic Computing: Knowledge Graph Empowers Artificial General Intelligence",
  	year="2023",
  	publisher="Springer Nature Singapore",
  	address="Singapore",
  	pages="3--16",
  	isbn="978-981-99-7224-1"
    }												
  					

    Recent studies have shown that subgraphs of the head entity, such as related relations and neighborhoods, are helpful for Knowledge Graph
    Reasoning (KGR). However, prior studies tend to focus solelyon enhancing entity representations using related relations, with little attention 
    paid to the impact of different relations on different entities and their importance in various reasoning paths. Meanwhile, conventional Graph
    Neural Networks (GNNs) utilized for KGR consider simultaneously neighboring nodes and connected relations of the head entity but typically 
    use a standard message-passing paradigm over the entire Knowledge Graph (KG). This results in oversmoothed representations and limits 
    efficiency. To address the above-mentioned limitations of existing methods, we propose a Dynamic Weighted Neural Bellman-Ford Network 
    (DyNBF) for KGR, which utilizes relation weights generated from subgraphs to compute only the most relevant relations and entities. This 
    way, we can integrate multiple reasoning paths more flexibly to achieve better interpretable reasoning, while scaling more easily to more
    complex and larger KGs. DyNBF consists of two key modules: 1) a transformer-based relation weights generator module, which computes
    the weights of different relations on the path with a sequence-to-sequence model, and 2) an NBFNet-based logic reasoner module, which 
    obtains entity representations and conducts fact prediction with dynamic weights from the previous module. Empirical results on three stand-
    ard KGR datasets demonstratethat the proposed approach can generate explainable reasoning paths and obtain competitive performance.
  			

Copyright© 2024 Huanxuan Liao(廖桓萱).