As a legal technology company, we at Blue J are always excited about new advancements in AI technology like GPT-4. It has the potential to revolutionize the way we practice law, providing faster and more accurate research, analysis, and predictions. However, it's important to approach new tools like GPT-4 with a sense of responsibility and caution, especially when it comes to complex and nuanced areas of law like taxation.
The GPT-4 demo earlier this week provided an interesting example of how this tool can be used to quickly retrieve information from legal sources, but it also highlighted the need to carefully tailor these tools using expert knowledge in order to arrive at accurate results in specialized areas. In the case of taxation, large language models will produce the best results when developed and used by tax experts who are well-versed in the complexities of tax law and the various sources of information that must be considered.
GPT-4 Developer Livestream Demo
During the GPT-4 Developer Livestream demo, OpenAI showcased the capabilities of GPT-4 to read and apply the tax code to a fictional fact pattern while explaining its reasoning. To set up the problem, approximately 16-pages from relevant sections of the Internal Revenue Code were provided in the prompt as context by the user. The model was then asked to compute the standard deductions for 2018. In explaining its work, the model correctly identified that there are special rules for 2018 to 2025 that increased the available standard deductions.
Next, the demonstrator asked the model to compute the tax liability of the married couple in the example. Unfortunately, the model did not identify the proper tax rates that the Code imposes for 2018 through 2025. As a result, the tool calculated the married couple’s tax 2018 liability using tax tables that are decades old and produced an inaccurate result.
GPT-4 and Taxation: Importance of Expertise
This minor oversight demonstrates that even seemingly simple tax questions can be complex and nuanced. To its credit, GPT-4 provided the citation to § 1(a)(2) of the Internal Revenue Code (the Code), which allowed experts to check its work and diagnose the problem. However, in performing its analysis, GPT-4 overlooked the fact that § 1(j)(2)(A) overrides § 1(a)(2) for the years 2018-2025 and contains updated tax brackets introduced by the Tax Cuts and Jobs Act in 2017 (TCJA). This error highlights the importance of taking a comprehensive approach to tax research and analysis.
GPT-4 shows an impressive ability to read and parse complex provisions of the Code. As background, § 1 of the Code imposes tax on individuals, setting forth applicable tax rate tables. Subsection 1(a) facially applies to married individuals filing joint returns and surviving spouses. Paragraph (2) of that subsection states that if income is over $36,900 but not over $89,150, the tax imposed will be $5,535, plus 28% of the excess over $36,900. This is the provision GPT-4 used to calculate a tax liability of $10,597.68 for a couple with a taxable income of $54,981.
However, an experienced tax professional would recognize that subsection 1(a) is only the beginning of the analysis. As every tax practitioner knows, you need to keep reading. Further along in § 1 is subjection (j), introduced by the TCJA. That subsection contains an updated rate table and states that it must be applied in lieu of the rate table in subsection (a) for tax years 2018-2025. According to this new rate table, the applicable tax on the first $19,050 of taxable income in 2018 is $1,905, plus 12% of the excess over $19,050. In the case of the couple with a taxable income of $54,981 (and no other relevant facts), the tax liability for 2018 is actually $6,216.72, which is $4,380.96 less than the liability calculated by GPT-4 during the demo.
This example highlights the importance of looking beyond a single section of the Code and taking into account all relevant sources of information. In fact, the IRS issues revenue procedures on a yearly basis that make cost of living adjustments to the tax brackets. These adjustments don't show up in the Code itself, further complicating the task of determining tax brackets. For anyone filing their returns for the 2022 tax year, Rev. Proc. 2021-45 sets forth the inflation-adjusted items applicable to various Code provisions for that year, including modifications to the tax rate tables under § 1(j)(2)(A). This means that determining the appropriate tax brackets for any given year requires information retrieval from multiple sources, which is time-consuming, and can be a challenge for anyone who is not a seasoned tax expert.
Collaboration between AI Technology and Human Expertise
While AI-powered tools may seem like a tempting solution for tax advice, it's important to keep in mind that they are only as reliable as the expert knowledge and analysis that goes into building them. To use AI-powered tools effectively for tax advice, it's essential for tax advisors to work with experienced tax professionals who can ensure the tool is properly tailored and the information retrieved is accurate and up-to-date.
At Blue J, we take a responsible approach to leveraging large language models like GPT-4 and have developed a tool for tax practitioners called Ask Blue J. In building Ask Blue J, we prioritize collaboration between AI technology and human expertise, combining the power of AI with the knowledge and experience of our tax professionals to provide accurate and reliable tax information to our customers. Any company developing tax tools using large language models like GPT-4 must take a carefully tailored approach to ensure their accuracy and reliability. This requires leveraging tax expertise to properly process the information and consider all relevant sources for a particular tax issue.
Blue J is thrilled to be at the forefront of harnessing the potential of AI-powered legal technology to transform legal practice.
To join the waitlist for Ask Blue J, sign up here: https://www.bluej.com/ask
Stay up to date
