Programming

Template-based PDF Document Generation in Java

Spread the love


In my previous article, I wrote about how we can seamlessly generate template-based documents in our javascript application using EDocGen. 

As a Java developer, I have worked on creating backend applications for e-commerce and healthcare enterprises where the requirement for document generation is vast. And they have mostly done in java spring boot application. This made me think, why not write an article on the Template-Based PDF Document Generation in Java? 

This is part II of my article “Template-Based PDF Document Generation in JavaScript“.

Let’s get into the project. I’m going to write a Spring boot application.

Requirement:

This is the requirement I have taken for demonstration and I kept them simple enough.

  • The front end calls our backend endpoint with the input parameters and template ID.
  • Document generation should happen behind the screen and the front end should receive an immediate acknowledgement.
  • When the document is ready, it should be sent to the user over email. The mail id should be sent to the server as an optional query parameter. 

Project Setup

When setting up the spring boot application, the spring initializer comes in handy.

Step 1: 

  • Go to https://start.spring.io/
  • Configure project name and package details.
  • Add 
    • Spring starter web
    • Lombok – Annotations for boilerplate code, 
  • Generate. 

Spring initializer

Step 2:

  • Import the project into your IDE and soon after the dependencies with start downloading. If not you can kick it using MVN install.
  • Add commons-collections4 for the token cache using the expiring map.
    • <dependency>
          <groupId>org.apache.commons</groupId>
          <artifactId>commons-collections4</artifactId>
          <version>4.4</version>
      </dependency>

  • Once dependencies are loaded. Run the Application.java file to start the web server.

Now we have our project base set up is ready.

Packages and Classes:

  • DocumentController – Handle All document-related API calls
  • LoginService – Handle token generation for eDocGen
  • GeneralDocumentService – Handles document-related services from DocumentController
  • EmailService – Handles sending emails to the user
  • Other services, DTO and exceptions can be found here.

File structure

Login

In order to generate the documents, we need an access token to hit the eDocGen’s API. And I’m going to cache the token for 20 mins and regenerate it again. 

You should be asking me What kind of cache I’m using. I’m gonna use an in-memory cache using PassiveExpireMap from apache’s common-collections4. We can use EH cache (in-memory) or Radis (Distributed) and there are broadly used. But PassiveExpireMap is simple to use.

This process has 2 steps.

  1. Token generation
  2. Token Cache

Token Generation

Login service is responsible for token generation. As per the Single Responsibility Principle, the Login service is only responsible for token generation.

API : https://app.edocgen.com/login 

Request Body : 

{
    username : "<username>",
    password : "<password>"
}

Code Sample:

@Slf4j
@Service
public class LoginService {

    // restTemplate has been defined as a bean
    @Autowired
    private RestTemplate restTemplate;

    public static String urlLogin = "https://app.edocgen.com/login";
    public static String bodyLogin = "{ \"username\": \"YOUR USERNAME\", \"password\": \"YOUR PASSWORD\"}";

    public String login() {
        HttpHeaders headers = HeaderBuilder
                                .builder()
                                .accept(MediaType.APPLICATION_JSON)
                                .contentType(MediaType.APPLICATION_JSON)
                                .build();

        HttpEntity<String> requestEntity = new HttpEntity<>(bodyLogin, headers);
        String token;
        try {
            ResponseEntity<LoginResponse> responseEntity = restTemplate.exchange(urlLogin, HttpMethod.POST, requestEntity, LoginResponse.class);
            token = responseEntity.getBody().getToken();
        } catch (Exception e) {
            log.error("Failed to get the access token.", e);
            throw new LoginException("Failed to get the access token. Please check your username and password.");
        }
        return token;
    }
}

Token Cache

The TokenCache has been written to consume LoginService. TokenCache is only responsible for caching the token. When the token is expired, it needs to call LoginService to re-generate the token.

@Slf4j
@Component
@RequiredArgsConstructor
public class TokenCache {

    private static String CACHE_KEY = "x-access-token";

    private static PassiveExpiringMap<String, String> tokenCache;

    static {
        // token will be cached for 20 mins
        tokenCache = new PassiveExpiringMap<>(20 * 60 * 1000);
    }

  	@Autowired
    private final LoginService loginService;

    public String getToken() {
        String accessToken = tokenCache.get(CACHE_KEY);
        // After 20 mins, cached token will be removed from the map
        if(StringUtils.isEmpty(accessToken)) {
            synchronized (this) {
                accessToken = loginService.login();
                tokenCache.put(CACHE_KEY,accessToken);
            }
        }
        return accessToken;
    }
}

HeaderUtils

Every time we hit the API we need to attach a set of headers to the request like an access token, and content type. 

This header Utils take care of token generation from the token cache and attaching other parameters.

package com.api.edocgen.util;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Component;

@Component
public class HeaderUtils {

    @Autowired
    private TokenCache tokenCache;

    public HttpHeaders buildHeaders() {
        return HeaderBuilder
                .builder()
                .accept(MediaType.APPLICATION_JSON)
                .contentType(MediaType.MULTIPART_FORM_DATA)
                .accessToken(tokenCache.getToken())
                .build();
    }

}


public class HeaderBuilder {

    private MediaType accept;
    private MediaType contentType;
    private String token;
    private final HttpHeaders headers = new HttpHeaders();
    private static final String ACCESS_TOKEN = "x-access-token";

    public static HeaderBuilder builder() {
        return new HeaderBuilder();
    }

    public HeaderBuilder accept(MediaType accept) {
        this.accept = accept;
        return this;
    }

    public HeaderBuilder contentType(MediaType contentType) {
        this.contentType = contentType;
        return this;
    }

    public HeaderBuilder accessToken(String token) {
        this.token = token;
        return this;
    }

    public HttpHeaders build() {
        if(Objects.nonNull(accept))
            headers.setAccept(Collections.singletonList(accept));
        if(Objects.nonNull(contentType))
            headers.setContentType(contentType);
        if(Objects.nonNull(token))
            headers.add(ACCESS_TOKEN, token);
        return headers;
    }

}

Document Templates

Basically, this article is about Generating documents based on pre-defined templates. These templates should follow the rules defined by eDocGen. Let me explain them at a high level.

Dynamic Text fields:  When we have the text fields that need to be shown in the document, it needs to be included in the template enclosed by {} .
For example, I need to print the name of the person in the document, {full_name} which needs to be included in the template.

Tables: A table is used for displaying the list of rows/records. It starts with <#listname>and ends with </listname>

Conditional Data: There are times we need to show the data based on the user, where conditions can be used in the documents. The syntax is {#fieldnmae="data"} followed by {/} to mark the end of the conditional statement.
For example,

{#country="INDIA"}
	INR: {priceinr}{/}
{#country="US"}
	$ {priceusd}{/}

The users from India will see the price calculated in INR, whereas the users from the US will see the price calculated in USD.

Mathematical Formula:

eDocGen provides support for all kinds of mathematical operations like +, -, *, / and priorities can be defined with ()

For example,

{((price_1*qauntity_1)+tax_1)+((price_2*qauntity_2)+tax_2)} 

You can find more details here

I have created one template and uploaded it to eDocGen. I’m going to use this template for our demo.

Data to Document generation:

Our requirement is that we need an API that accepts JSON or XML data files and processes the request asynchronously and emails the generated document to the given email.

Let’s start from the Service layer and then the Controller layer.

DocumentService:

Document Service will be handing the Input parameters from the controller. The data source for the input data could be either a file or DB Parameter. We will have two branches in code one will handle the DB Parameters and the other will handle the input file.

For the document generation, we use the eDocGen’s /api/v1/document/generate/bulk API.

1. DocumentGeneration – For JSON/XML files

For document generation with JSON/XML files as the data sources, we will be using form-data.

Base URL https://app.edocgen.com/
Method POST
Path /api/v1/document/generate/bulk
Parameters documentId id of the template
format pdf/doc (Any Format that are supported by the template)
outputFileName The name for the output file.
inputFile json, xlsx and xml supported.
Headers Content-Type multipart/form-data
x-access-token Json web token for access

Code Implementation:

/**
     * Gets the documentId, output format, file data as resource, email id of the recipient
     * and the mode of the document generation.
     * When the request is made for bulk generation the all the files will be zipped after generation
     * and mailed.
     * @param documentId
     * @param outputFormat
     * @param resource
     * @param email
     * @param isBulkMode
     * @throws Exception
     */
    public void getDocByApi(String documentId, String outputFormat, ByteArrayResource resource, String email, boolean isBulkMode) throws Exception {
        RestTemplate restTemplate = new RestTemplate();
        String outputFileName = UUID.randomUUID().toString();
        try {
            //create the body request
            MultiValueMap<String, Object> body = createBody(resource, documentId, outputFileName, outputFormat);
			
            //set Headers
            HttpHeaders headers = headerUtils.buildHeaders();
            //send the request to generate document
            HttpEntity requestEntity = new HttpEntity(body, headers);

            ResponseEntity<String> generateResponse = restTemplate.postForEntity(urlBulkGenerate,
                    requestEntity, String.class);
            if (HttpStatus.OK == generateResponse.getStatusCode()) {
                processOutput(outputFileName, outputFormat, !isBulkMode, email);
            }
        } catch (Exception e) {
            log.error("Error in the generating document", e);
        }
    }


   private MultiValueMap<String, Object> createBody(ByteArrayResource resource,
                                                     String documentId, String outputFileName, String outPutFormat) {
        MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
        body.add("documentId", documentId);
        body.add("inputFile", resource);

        body.add("format", outPutFormat);
        body.add("outputFileName", outputFileName);
        // to download directly the file
        body.add("sync", true);
        return body;
    }

2. Documentgeneration – For Database Query as Source

This is much similar to the file as input data, but this will have a different set of parameters.

Base URL https://app.edocgen.com/
Method POST
Path /api/v1/document/generate/bulk
Parameters documentId id of the template
format pdf/docx (Format should be supported by the template)
outputFileName The file name for the output file.
dbVendor MySql/Oracle/Postgresql
dbUrl JDBC Url
dbLimit Number of rows to be limited
dbPassword Database password
dbQuery Query to be executed
Headers Content-Type multipart/form-data
x-access-token JWT auth token from login

The Code Implementation:

/**
     * Collects the database parameters and submit the request to generate the document.
     * Calls processOutput that will take care of sending the document over email
     * @param documentId
     * @param outputFormat
     * @param dbparmeters
     * @param email
     * @throws Exception
     */
    public void getDocsByDatabase(String documentId, String outputFormat, DBParameters dbparmeters, String email) throws Exception {

        String outputFileName = UUID.randomUUID().toString();

        MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
        body.add("documentId", documentId);
        body.add("format", outputFormat);
        body.add("outputFileName", outputFileName);
        body.add("dbVendor", dbparmeters.getDbVendor());
        body.add("dbUrl", dbparmeters.getDbUrl());
        body.add("dbLimit", dbparmeters.getDbLimit());
        body.add("dbPassword", dbparmeters.getDbPassword());
        body.add("dbQuery", dbparmeters.getDbQuery());

        // set Headers
        HttpHeaders headers = headerUtils.buildHeaders();

        // send the request to generate document
        HttpEntity requestEntity = new HttpEntity(body, headers);
        try {
            ResponseEntity<String> generateResponse = restTemplate.postForEntity(urlBulkGenerate, requestEntity, String.class);
            log.info("Send to edocGen Response : " + generateResponse.getStatusCode());
            if (HttpStatus.OK == generateResponse.getStatusCode()) {
                processOutput(outputFileName, outputFormat, false, email);
            }
        } catch (Exception e) {
            log.error("Failed to get the file downloaded");
        }

    }

Now you should be asking about the role of the processOutput method. 

It basically takes the name of the file and searches whether the file is generated or not in eDocGen. Once documents are generated successfully, it triggers the send the file over email API. The emailing functionality has a dedicated service called EmailService.

For searching the file we will be using eDocGen’s /api/v1/output/name/{output_file_name}

Base URL https://app.edocgen.com/
Method GET
Path /api/v1/output/name/{output_file_name}
Headers Content-Type multipart/form-data
x-access-token JWT auth token from login

When the file is successfully generated the API returns the output-id of the generated document which will be used for the sending file over email.

Code Implementation:

/**
     * 
     * @param outputFileName
     * @param outputFormat
     * @param isSingleGeneration
     * @param email
     */
    private void processOutput(String outputFileName, String outputFormat, boolean isSingleGeneration, String email) {
        HttpHeaders headers = headerUtils.buildHeaders();
        HttpEntity requestEntity = new HttpEntity(null, headers);

        ResponseEntity<OutputResultDto> result = restTemplate.exchange(baseURL + "output/name/" + outputFileName + "." + outputFormat, HttpMethod.GET, requestEntity, OutputResultDto.class);
        OutputDto responseOutput = null;

        if (!isSingleGeneration) {
            outputFormat = outputFormat + ".zip";
        }

        responseOutput = isFileGenerated(outputFileName, outputFormat, requestEntity, result);

        log.info("Output Document Id generated at edocgen is : " + responseOutput.get_id());
        String outputId = responseOutput.get_id();

        // output download
        try {
            emailService.sendOutputViaEmail(outputId, email);
            log.info("File has been sent over email for file : " + outputFileName + "." + outputFormat);
        } catch (Exception e) {
            log.error("Error while Sending the file over email");
            throw new FileNotGeneratedException("Error while Downloading File");
        }
    }

    private OutputDto isFileGenerated(String outputFileName, String outputFormat, HttpEntity requestEntity, ResponseEntity<OutputResultDto> result) {
        OutputDto responseOutput;
        int retryCounter = 0;
        try {
            while (result.getBody().getOutput().toString().length() <= 2 && retryCounter < 200) {
                log.info("Output file is still not available. Retrying again!!!! Counter : " + retryCounter);
                result = restTemplate.exchange(baseURL + "output/name/" + outputFileName + "." + outputFormat, HttpMethod.GET, requestEntity, OutputResultDto.class);
                retryCounter++;
                // spin lock for 500 milli secs
                Thread.sleep(5000);
            }
            responseOutput = result.getBody().getOutput().get(0);
        } catch (Exception error) {
            log.error("Error : Output file is not available after 200 tries");
            throw new FileNotGeneratedException("Error : Output file is not available after 200 tries");
        }
        return responseOutput;
    }

EmailService:

Email service takes the output ID of the document and the email id to which the document needs to be sent.

For sending document over email, we will be using /api/v1/output/email from eDocGen.

Base URL https://app.edocgen.com/
Method POST
Path /api/v1/output/name/{output_file_name}
Headers Content-Type multipart/form-data
x-access-token JWT auth token from login

Code implementation: 

package com.api.edocgen.service;

import com.api.edocgen.util.HeaderUtils;
import com.api.edocgen.util.TokenCache;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service;
import org.springframework.util.LinkedMultiValueMap;
import org.springframework.util.MultiValueMap;
import org.springframework.web.client.RestTemplate;

@Slf4j
@Service
public class EmailService {

    public static String urlOutputEmail = "https://app.edocgen.com/api/v1/output/email";

    @Autowired
    private RestTemplate restTemplate;

    @Autowired
    private HeaderUtils headerUtils;

    public void sendOutputViaEmail(String outId, String emailId) {
        RestTemplate restTemplate = new RestTemplate();
        try {
            MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
            body.add("outId", outId);
            body.add("emailId", emailId);
            // set Headers
            HttpHeaders headers = headerUtils.buildHeaders();
            // send the request to generate document
            HttpEntity requestEntity = new HttpEntity(body, headers);

            ResponseEntity<String> generateResponse = restTemplate.postForEntity(urlOutputEmail, requestEntity, String.class);

            log.info("Send to edocGen Response : " + generateResponse.getStatusCode());
            if (HttpStatus.OK == generateResponse.getStatusCode()) {
                log.info("Email sent");
            }
        } catch (Exception e) {
            log.error("Exception During Sending Email. Check if document id is valid", e);
        }
    }

}

DocumentController: 

Great. Now we have our backend logic. Let’s expose the logic with an API. 

The source for the document generation can be either a JSON or XML file or can be a Database query with connection details.

So we need to design a controller that accepts both JSON(for DB parameters) and FormData (for file upload).

Method POST Required Default
Path /document/{document_id}/{email}
Parameters document_id Yes
email Yes
is_bulk No TRUE
output_file_format No PDF
file Yes,
when DB parameters are not available
dbParameters Yes,
when the input file is not available
Accepts multipart/form-data
(or)
application/json
Produces application/json

The code implementation:

@RequestMapping("/document")
@RestController
@RequiredArgsConstructor
public class DocumentController {


    private final DocumentService documentService;

    /**
     * 
     * This method handles FormData with file input
     * @param documentId
     * @param email
     * @param isBulk
     * @param format
     * @param file
     * @return
     */
    @PostMapping(value = "/{document_id}/{email}",
            consumes = {
                    MediaType.MULTIPART_FORM_DATA_VALUE
            },
            produces = MediaType.APPLICATION_JSON_VALUE )
    public ResponseDto generateDocument(@PathVariable("document_id") String documentId,
                                        @PathVariable String email,
                                        @RequestParam(value = "is_bulk", required = false, defaultValue = "true") boolean isBulk,
                                        @RequestParam(value = "output_file_format", required = false, defaultValue = "pdf") String format,
                                        // Accept any type of file
                                        @RequestPart(value = "file", required = false) MultipartFile file) {
        if(Objects.isNull(file)) {
            return new ResponseDto("Input is empty");
        }
        try {
            documentService.getDocByApiAsync(documentId, format, file, isBulk, email, null);
        }catch (Exception e) {
            return new ResponseDto("Your request has been failed. Please check your input", HttpStatus.BAD_REQUEST);
        }
        return new ResponseDto("Your request has been submitted. You will receive the email with document");
    }

    /**
     * This method handles Application/json content type
     * @param documentId
     * @param email
     * @param dbParameters
     * @param isBulk
     * @param format
     * @return
     */
    @PostMapping(value = "/{document_id}/{email}",
            consumes = {
                    MediaType.APPLICATION_JSON_VALUE
            },
            produces = MediaType.APPLICATION_JSON_VALUE )
    public ResponseDto generateDocumentForDb(@PathVariable("document_id") String documentId,
                                        @PathVariable String email,
                                        // Handle dbParameters
                                        @RequestBody(required = false) DBParameters dbParameters,
                                        @RequestParam(value = "is_bulk", required = false, defaultValue = "true") boolean isBulk,
                                        @RequestParam(value = "output_file_format", required = false, defaultValue = "pdf") String format
                                        ) {
        if(Objects.isNull(dbParameters)) {
            return new ResponseDto("Input is empty");
        }
        try {
            documentService.getDocByApiAsync(documentId, format, null, isBulk, email, dbParameters);
        }catch (Exception e) {
            return new ResponseDto("Your request has been failed. Please check your input", HttpStatus.BAD_REQUEST);
        }
        return new ResponseDto("Your request has been submitted. You will receive the email with document");
    }

}

Time for Demo:

JSON to Document:

Now we are ready to test our application.

I’m hitting the /document/{document_id}/{email} API we have created now with a JSON file as input data.

Please take a look at the postman configuration.

postman configuration

After hitting the API, we instantly got the response and the file is being processed in the background.

The input JSON for your reference:

[{
    "Invoice_Number": "SBU-2053502",
    "Invoice_Date": "31-07-2020",
    "Terms_Payment": "Net 15",
    "Company_Name": "Company B",
    "Billing_Contact": "B-Contact2",
    "Address": "Seattle, United States",		  
    "Logo":"62b83ddcd406d22dc7516b53", 
    "para": "61b334ee7c00363e11da3439", 
    "Email":"test@gmail.com",
    "subtemp": "62c85b97f156ce4fbdb01bcb",
    "ITH": [{
      "Heading1":"Item Description",
      "Heading2": "Amount"

    }],
    "IT": [{	       
      "Item_Description": "Product Fees: X",
      "Amount": "5,000"

    },
           {	       
             "Item_Description": "Product Fees: Y",
             "Amount": "6,000"

           }]
		}
]

Once the file is processed and the final documents are generated by eDocGen, we got the attachment sent via email.

 attachment sent via email

XML to Document:

Now let’s try with XML as an input file with the same data and postman configuration.

XML to Document:

And we got the files sent via email by eDocGen.

And that brings us to the end of this article. Hope you enjoyed reading it.



Source link

Related Articles

Leave a Reply

Your email address will not be published.

Back to top button