Richard Pajerski  Software development and consulting

And the Word was made flesh, and came to dwell among us (John 1:14, Knox Version).

Image (public domain): Our faith and its defenders: comprising the trials and triumphs of the defenders of our faith in America by John Gilmary Shea.

is right around the corner (August 7, 2019):  https://register.gotowebinar.com/register/5645329603541153035. 

Looking forward to hearing about the roadmap and how to sign up for the beta.

Be they public, private or the newly-minted "Partner-led" clouds, one message HCL is making clear is that the future for the newly-acquired IBM collaboration product portfolio will be all about the cloud.  Richard Jefts (General Manager, HCL Digital Solutions) has just published here on the new approach: Update on HCL Acquisition of IBM Collaboration Portfolio (Edit March 2021: Update on HCL Acquisition of IBM Collaboration Portfolio).

It appears that the biggest impact will be on current SmartCloud Notes users who will now need to transition away from that offering to one of the new HCL cloud models.  In practice, that will probably entail some form of migration back to a traditional, on-premises Domino/Portal solution (or Domino/Portal-hosted partner solution).  Those who currently run on-premises applications and solutions, including Verse, will (likely) not be immediately affected.

It's good to hear mail will continue to be a core area for Domino; however, the "for the foreseeable future" is an interesting qualification that suggests it may not be for long.  The tight integration of email within collaborative Notes/Domino applications won't go away but will loosen up as different messaging providers take over the role of Notes mail clients.  HCL's announcement here is not surprising (or at least not shocking) and I think it strikes the right balance between managing what works well today on-premises and where much new development will be going forward: cloud.

Having developed with Java for a number of years in various environments (Notes/Domino, Tomcat, ActiveMQ,  Android, desktop, etc.), I was initially skeptical when I read this article and watched the video about the recently-improved Eclipse OpenJ9: https://developer.ibm.com/videos/introduction-to-eclipse-openj9-and-adoptopenjdknet/

Yes, Java has incrementally improved over time but the claims here seemed a bit over the top.  To think I might get both noticeably faster startup *and* up to 50% memory reduction just by switching to J9 seemed to be a bit too optimistic.  But after downloading (adoptopenjdk.net) and giving it a spin, I was not disappointed.

Sure enough, out-of-the-box startup time for Netbeans 8.2 on Windows 8.1 increased dramatically against Oracle Java 1.8.0_191 (running quad-core I7 on SSD).  There was no point in taking measurements -- it was up and ready in three seconds!  This didn't seem possible with Netbeans but there it was.  Everything worked the same as before ... only faster.  Then the real shocker: RAM usage went from roughly 650M down to 268!  Huh?  If I can eliminate that much RAM usage for hosted server side deployments, it's going to translate into real cash savings.

On top of the performance upgrade and memory savings, I immediately noticed that Swing is visually better in J9 than OpenJDK [edit: with the HotSpot VM].  In particular, the default font rendering is really nice!  In the past, OpenJDK has generally lagged behind Oracle Java for desktop applications and still does; but to my eyes, J9 is now at visual parity with Oracle (or perhaps better).

I realize that the J9 has been the JVM in Notes/Domino all these years but I've never attempted to benchmark it against other JVMs since IBM never really promoted it as a JDK for Windows.  I'm currently using 9.0.1 FP10 which uses build 8.0.5.21 of J9:

Hopefully, IBM can manage to get the latest J9 into an upcoming fixpack.  I sure have lots of Notes and Domino Java code that could benefit from it.

A big congratulations and thank you to Mark Stoodley and all the other engineers and players behind this release!

[Edit February 2020: the LEND product has been renamed to CertMatica]

Version 2.0 of LEND is now available and comes with domain wildcard certificate support via DNS challenge.

Wildcard certificates are convenient particularly in situations where a single Domino server hosts multiple virtual sites, each of which needs SSL/TLS protection. Managing separate certificates for each Domino SSL site in this situation is feasible but not very practical since each one requires its own IP address.  A wildcard certificate takes care of that issue and fortunately, Let's Encrypt began offering wildcard certificates earlier this year.  However, as of this blog posting, they're only supported with the DNS-01 challenge type.

The DNS challenge feature was interesting to implement because Let's Encrypt DNS challenges do not offer the same level of automation as HTTP challenges.  With the DNS challenge, Let's Encrypt servers will query your hosting provider during the challenge/response phase instead of your HTTP server (which is queried when using the HTTP challenge).  Since there's no industry-standard way to modify DNS records, the challenge must be entered manually at renewal time, typically using your hosting provider's custom web interface. Fortunately, LEND now has built-in workflow to remind administrators when to do so at renewal time!

Take LEND for a test ride and let me know what you think.

Image (public domain): Philippe de Champaigne

 

Among the areas getting attention in the upcoming release of Domino 10 is the data store.  As part of making Domino more bullet-proof, removing the 64GB limit on the NSF size is planned and will be a fantastic improvement.  Naturally, this is going to put more emphasis on overall database scalability.  As Domino gets more scalable, I think it's going to need a feature that many of us have been seeking for some time: multi-document ACID transactions.

Being able to save two or more documents as a single transaction cannot currently be done natively in Domino.  The best we can get is saving all of the fields associated with a single document using NotesDocument.Save: the operation either succeeds or fails.  This is fine for most Notes/Domino applications but there are a number of cases where having a transactional save across multiple documents is desirable.  A simple work order system where multiple, related tasks are attached to a main work order request is an obvious example.  Implementing each task as a separate document is an intuitive approach and can simplify programming of such a system.

MongoDB 4.0 recently introduced multi-document transactions.  And although Domino doesn't directly compete with MongoDB or other NoSQL databases, as it begins to scale, it's nonetheless going to need to act more like them.

What do you think?

By default, the James Mail Server (3.x versions) saves a copy of all outbound messages on the server. While this may be appropriate in certain scenarios, it doesn't seem to be a good default setting for your average POP3/SMTP installation. All of that sent mail accumulates on the server without any apparent mechanism for removing it. Deselecting the "Leave messages on server" setting found in POP3 mail clients does not apply here since outbound mail is sent via SMTP.

To disable this feature, comment out or remove the following block in mailetcontainer.xml (and restart):

<-- Place a copy in the user Sent folder -->
<mailet match="SenderIsLocal" class="ToSenderFolder">
   <folder>Sent</folder>
   <consume>false</consume>
</mailet>

IBM has announced a multi-year investment in Notes Domino with a major new release (Notes 10) coming out in 2018. The investment will include closely-related products such as Notes Traveler, IBM Sametime and IBM Verse.

Specific product details are scant at the moment but it's encouraging to see IBM laying out a long-term roadmap for Notes and Domino and broadcasting a commitment to protecting its clients' investments. Also, the new direction allows IBM partners to hope for commercial stability for these products for the foreseeable future. Overall, I'm optimistic about this announcement.

However, the partnership with HCL Technologies for future development raises some questions. Will HCL Technologies be able to innovate the way that Iris Associates once did? Is this merely a cost-cutting measure or does IBM no longer have the internal talent to take Notes/Domino into the future (or both)?

More here:
https://www.ibm.com/blogs/social-business/2017/10/25/ibm-announces-investment-notes-domino-version-10-beyond/

See "IBM Domino Community Server for Non-Production Environments" here: https://www.ibm.com/developerworks/develop/collaboration/

This is the full Domino server product available at no charge. However, the restrictions are:
1) You have to select the "Utility Server" option (no mail).
2) It may only be used for testing applications in a non-production environment.

The latest feature pack (FP9 as of September 15, 2017) is also available for download.

*** July 2019 Update ***  Domino Community Server version 10.0.1 FP2: https://www.ibm.com/account/reg/us-en/signup?formid=urx-33713

*** July 2018 UPDATE ***
Links to the final database and Java source are included at the bottom of this blog entry.

In Parts 1 and 2, we created a Java applet and incorporated it into a Notes database. In this last part, we're going to code the export functionality in our applet and refresh the applet in Notes.

===================================
Part 3 -- Export Notes data using the applet
===================================

Step 1. Program the Start button. Open the ContactExport class in NetBeans and at the top of the code editor, click the Design button for the visual representation of the applet. Then right-click on the Start button and then Events > Action > actionPerformed. NetBeans will create a method called jButton1ActionPerformed and place our cursor at the appropriate spot in the source where we can add code that executes when the button's clicked. So we start with:

private void jButton1ActionPerformed(java.awt.event.ActionEvent evt) {                                         

        // TODO add your handling code here:

    }    

When we click the Start button, we want to immediately disable it so only one export operation runs at a time. This would normally just be the following one line of code:


jButton1.setEnabled(false);


However, in Swing, we need to make sure to put any code that updates the UI on the event dispatch thread. So we're going to disable the button in a new thread:

private void jButton1ActionPerformed(java.awt.event.ActionEvent evt) {       

        // TODO add your handling code here:

        try {

            java.awt.EventQueue.invokeAndWait(new Runnable() {

                public void run() {

                    jButton1.setEnabled(false);

                }

            });

        } catch (Exception ex) {

            ex.printStackTrace();

        }

}

Step 2. Create a separate thread to access our Notes database. Next, we're going to create a separate thread that will connect to Notes. For now, we're going to implement this as a private inner class inside our applet. Toward the bottom of our ContactExample class, add:

private class ExportRunner implements Runnable {



        private Thread t;


        @Override

        public void run() {


        }


        public void start() {

            t = new Thread(this);

            t.start();

        }

    }

We're going to place most of our functional export code in the ExportRunner's run() method which is the method that gets called when we start a Java thread. Add the following block to the ExportRunner's run() method to access the "All Contacts" view and its documents:

        @Override

        public void run() {

                    try {

                        Session s = ContactExport.this.openSession();

                        Database db = s.getDatabase("", "Contacts.nsf");

                        View allContactsView = db.getView("AllContacts");

                        Document d = allContactsView.getFirstDocument();

                        while (d != null) {

                            // We access the data here...

                            Document tmpDoc = allContactsView.getNextDocument(d);

                            d.recycle();

                            d = tmpDoc;

                        }

                        allContactsView.recycle();

                        db.recycle();

                    } catch (NotesException nex) {

                        nex.printStackTrace();

                    }

         }

Step 3. Cycle through the contact documents and save the contents to internal buffer. First, add a StringBuilder in the ExportRunner class which will serve as the internal buffer to store the comma-delimited data. Also declare a platform-independent line separator.

private class ExportRunner implements Runnable {

        private Thread t;

        private StringBuilder sb = new StringBuilder();

        private String newline = System.getProperty("line.separator");

Back in ExportRunner's run() method, we next want to update the while loop to extract the contact field data and separate the fields with commas and a final carriage return at the end of each line:

                        while (d != null) {

                            String firstName = d.getItemValueString("FirstName");

                            String lastName = d.getItemValueString("LastName");

                            String phone = d.getItemValueString("Phone");

                            String city = d.getItemValueString("City");

                            sb.append(firstName);

                            sb.append(",");

                            sb.append(lastName);

                            sb.append(",");

                            sb.append(phone);

                            sb.append(",");

                            sb.append(city);

                            sb.append(newline);

                            Document tmpDoc = allContactsView.getNextDocument(d);

                            d.recycle();

                            d = tmpDoc;

                        }

Step 4. Update the progress bar as we go through the contacts. This may sound straightforward but it takes some setup. Let's start by adding these variables to the ExportRunner class, just below the StringBuilder declaration:

private class ExportRunner implements Runnable {

        private Thread t;

        private StringBuilder sb = new StringBuilder();

        private String newline = System.getProperty("line.separator");

        private int contactCount;

        private javax.swing.Timer progBarTimer;

NetBeans' GUI builder will have already declared a JProgressBar for us using the variable name jProgressBar1 when we added the progress bar component to our JPanel back in Part 1. But to animate the JProgressBar, we need to create two additional items: a handler class and timer class. We're going to create the handler class as a new private inner class in the ExportRunner class. We'll call it ProgressBarHandler:

private class ProgressBarHandler implements ActionListener {

            @Override

            public void actionPerformed(ActionEvent event) {

                int v = contactCount;

                if (v != 0) {

                    jProgressBar1.setValue(v);

                }

            }

        }

And we'll instantiate the Timer class by inserting this as our first line in the run() method of our ExportRunner class:


progressBarTimer = new javax.swing.Timer(2, new ProgressBarHandler());


The Timer takes our handler class as a parameter and, once started, will call its actionPerformed method every two milliseconds. In actionPerformed, we explicitly set the progress bar's value. When the thread starts, we start the Timer, prime the progress bar with starting and ending values and then update those values as we loop through the documents.

We're also going to add a new method to ExportRunner called isDone() that holds a boolean variable indicating when the export is complete. We set the boolean variable to true when we're done processing the documents and when the handler sees that, it sets the progress bar back to zero and stops the Timer. It also takes care of re-enabling our Start button.

Beyond the progress bar animation, a good GUI practice is to let the user know when the system is busy. We're going to do that with the following commands:


setCursor(Cursor.getPredefinedCursor(Cursor.WAIT_CURSOR));
setCursor(Cursor.getPredefinedCursor(Cursor.DEFAULT_CURSOR));


Place the WAIT_CURSOR command outside the ExportRunner class – at the beginning of the jButton1ActionPerformed method and put the DEFAULT_CURSOR command as the last line in the handler class.

Here's the latest version of ExportRunner:

    private class ExportRunner implements Runnable {



        private Thread t;

        private StringBuilder sb = new StringBuilder();

        private String newline = System.getProperty("line.separator");

        private int contactCount;

        private javax.swing.Timer progressBarTimer;

        private boolean done = false;


        @Override

        public void run() {


            try {

                progressBarTimer = new javax.swing.Timer(2, new ProgressBarHandler());


                Session s = ContactExport.this.openSession();

                Database db = s.getDatabase("", "Contacts.nsf");

                View allContactsView = db.getView("AllContacts");


                // Set the progress bar's minimum and maximum values.

                jProgressBar1.setMinimum(0);

                // Total contact documents in our view.

                jProgressBar1.setMaximum(allContactsView.getEntryCount());


                // Start the timer.

                progressBarTimer.start();


                Document d = allContactsView.getFirstDocument();


                while (d != null) {

                    Thread.sleep(1);    // Pause this loop to give other threads a chance to run.

                    contactCount++;     // Increment the number of contacts processed.


                    String firstName = d.getItemValueString("FirstName");

                    String lastName = d.getItemValueString("LastName");

                    String phone = d.getItemValueString("Phone");

                    String city = d.getItemValueString("City");

                    sb.append(firstName);

                    sb.append(",");

                    sb.append(lastName);

                    sb.append(",");

                    sb.append(phone);

                    sb.append(",");

                    sb.append(city);

                    sb.append(newline);

                    Document tmpDoc = allContactsView.getNextDocument(d);

                    d.recycle();

                    d = tmpDoc;

                }

                allContactsView.recycle();

                db.recycle();


            } catch (NotesException nex) {

                nex.printStackTrace();

            } catch (Exception ex) {

                ex.printStackTrace();

            } finally {

                done = true;

            }

        }


        public void start() {

            t = new Thread(this);

            t.start();

        }


        private boolean isDone() {

            return done;

        }


        private class ProgressBarHandler implements ActionListener {


            @Override

            public void actionPerformed(ActionEvent event) {

                int v = contactCount;

                if (v != 0) {

                    jProgressBar1.setValue(v);

                }

                if (ExportRunner.this.isDone()) {

                    progressBarTimer.stop();

                    jProgressBar1.setValue(0);

                    jButton1.setEnabled(true);

                    setCursor(Cursor.getPredefinedCursor(Cursor.DEFAULT_CURSOR));

                }

            }

        }

    }

Finally, to instantiate and run this new class when the Start button is clicked, go back to jButton1ActionPerformed and add these two lines to the run() method:

ExportRunner exportRunner = new ExportRunner();
exportRunner.start();


The actionPerformed method should now look like:

    private void jButton1ActionPerformed(java.awt.event.ActionEvent evt) {                                         

        // TODO add your handling code here:

        setCursor(Cursor.getPredefinedCursor(Cursor.WAIT_CURSOR));

        jButton1.setEnabled(false);

        ExportRunner exportRunner = new ExportRunner();

        exportRunner.start();

    }

Step 6. Store the output after prompting the user for a location. Our data is now in our StringBuilder object but in Java, it takes a few steps to get that data into a file. For additional flexibility, we're also going to prompt the user for a file name and location using a JFileChooser. Put this block just after the db.recycle() command in ExportRunner's run() method:

            JFileChooser jFileChooser = new JFileChooser();

            jFileChooser.setSelectedFile(new File("ExportedContacts.txt"));

            int save = jFileChooser.showSaveDialog(ContactExport.this);



            if (save == JFileChooser.APPROVE_OPTION) {

                FileOutputStream stream = null;

                PrintStream out = null;

                try {

                    File file = jFileChooser.getSelectedFile();

                    stream = new FileOutputStream(file);

                    out = new PrintStream(stream);

                    out.print(sb.toString());

                } catch (Exception ex) {

                    ex.printStackTrace();

                } finally {

                    try {

                        if (stream != null) {

                            stream.close();

                        }

                        if (out != null) {

                            out.close();

                        }

                    } catch (Exception ex) {

                        ex.printStackTrace();

                    }

                }

            }

That's it for the code. Now recompile the applet and refresh it in Notes (see Step 5 of Part 2). Populate the Notes database with some sample contact data and try it out.

Conclusion
Embedding custom Java applets in the Notes client is a powerful way of extending the value of Notes client applications. It's a little more involved than building out similar functionality with traditional Notes design elements but it offers benefits that just aren't available in Notes. Considering the size of the Java API and availability of third party Java-based tools, this example only scratches the surface of the possibilities.

Files and source code
Contacts.nsf.zip
ContactExport-src.zip

In Part 1, we created a Java applet using NetBeans that will allow us to export Notes contact data. In this second part, we're going to walk through getting that applet set up properly in a Notes database. I've created a small database called Contacts and populated it with several contact documents, each with only four fields: First Name, Last Name, Phone and City. In order to highlight the basic goal of applet integration, I'm only going to discuss the database design aspects directly related to the applet.

========================================
Part 2 -- Integrating the applet in the Notes Client
========================================

Step 1. Compile the project in NetBeans. First compile the applet (right-click on the project then Clean and Build). This step will produce a file called ContactExport.jar, which will be created in the project's dist folder.

Step 2. Create a Page element and embed the applet in it. In Domino Designer, create a new Page design element called ContactApplet. Open the page, hit Enter at least once to produce a small page header and then type Export Contact List. Hit Enter again and from the menu, Create > Java Applet. You'll first be presented with a Create Java Applet dialog; click the Locate button in the lower right to get the Locate Java Applet Files dialog. Now click the folder icon beside the Base Directory field to navigate to our project's dist folder (in my case, D<colon><backslash>temp<backslash>ContactExport<backslash>dist). Click Ok and you should now see the ContactExport.jar file. Click Add/Replace File(s) and in the Base class field at the top right, enter com.example.ContactExport.class (which is our applet's main class file). Click Ok twice and save the Page.

Step 3. Link the ContactExport Page element to the default Outline. The Contacts database uses a Frameset and Outline for the main navigation where clicking on the left-hand navigation places the view or page on the right frame. I've already created a default Outline called Main that displays the database's views and now we're going to add an entry to the Outline. The entry will be the ContactExport Page element from Step 2.

At this point, the applet is ready to work in the Notes client. However, we'll need to make some minor adjustments to both the applet and page to improve the presentation.

Step 4. Change applet look and feel and adjust border. For better visual integration, we're going to tell the applet to use the "system" look and feel so that it fits in better with the underlying platform (in this example, Windows 8). In the notesAppletInit method, expand the section labelled "Look and feel setting code". Remove the for{} block and replace it with:

javax.swing.UIManager.setLookAndFeel(javax.swing.UIManager.getSystemLookAndFeelClassName());

This tells Java to pick up the native look and feel for the underlying platform. In addition, since we're only going to be displaying a button and progress bar, we're also going to shrink the applet window size. Simply grab the frame of the applet in the GUI builder and drag to resize. Once you have an appropriate size, double-click the border and record the applet's width and height -- we'll need to use these dimensions for the embedded applet in the page design element.

Step 5. Update the page and resize the applet. Be sure to Clean and Build the project again in NetBeans so that a new applet jar file is created in the project's dist directory. Now go back to the ContactApplet page in Domino Designer, right-click on the embedded applet, click Refresh from the menu then Refresh All; click Ok and save the page.

Next, resize the applet in the page to match the dimensions from the applet's window size from NetBeans (Step 4 above). Unfortunately, this must be done with the mouse -- the size properties on the Java Applet properties box is read only.

Finally, I'm going to change the background color of the page to light gray to better blend with the applet and move applet slightly towards the left margin (Text Properties box).

And here's what we have so far:

In Part 3, we'll code the applet to select documents from the Notes view and export them to a comma-delimited file.

Notes/Domino application development has for years targeted the web browser for the GUI but for many organizations, a large portion of apps are written for the native Notes client. While a common path for extending these native apps is to develop a portion of them as web apps, another powerful way is to implement Java applets in forms and pages. Even though Java applets were initially intended for web browsers, we can use use them as though they were native Notes widgets right in the Notes client.

The ability to embed Java applets in the Notes client has been available since Java itself was introduced into the environment way back in the 4.x days. However, applets were a tough sell then for a number of reasons, among them slow load times and sub-par look and feel. It wasn't until Java 6 was available in the 8.x client – a dozen years later – that custom applets became a realistic option for production Notes apps.

Let's look at a custom applet that can retrieve a contact list from a Notes 9 database and let the user export that data to a comma-delimited file. For this example, I'm using the NetBeans IDE (version 8.02) to develop the applet. The applet will compile to a jar archive which we'll import into the Designer client. Familiarity with Java and using Domino Designer will be assumed but since NetBeans and Java Applets in particular are not commonly used in Notes development, I'll try to be a bit more detailed when referring to them.

In Part 1, we'll set up the applet project in NetBeans. In Part 2, we'll walk through getting the applet integrated in Notes. Finally, in Part 3, we'll code the applet source with the main functionality and then refresh the Notes database with the updates.


=================================
Part 1 -- Setting up the NetBeans project.
=================================

Step 1. Create a new Java Project. From the menu, Start > New Project and choose new Java Application. I'm calling the new project ContactExport and using com.example as the default package.

By default, this creates a class file called ContactApplet but we're not going to use it. We're going to create a JApplet Form (Step 2) file and use that instead so the ContactApplet file can safely be deleted.

Step 2. Create a JApplet. Create a new JApplet form in the com.example package called ContactExport. This allows us to use the GUI builder in NetBeans which can greatly simplify designing a Swing-based user interface.

Step 3. Set Source/Binary Format to JDK 6 and add NCSO.jar as a library. Right-click on the project, then Properties, Source and make sure that the Source/Binary Format (at the bottom of the dialog) is set to JDK 6 which matches the JRE version on Notes 9. Next, in the same dialog, click Librairies on the left-hand navigation and then on the right, Add JAR/Folder. Here we add the NCSO.jar (found in the Notes\Data\domino\java directory) as a library for this project to expose the Notes Java API.

Step 4. Extend JAppletBase. By default, the applet extends javax.swing.JApplet but let's change that to extend lotus.domino.JAppletBase. Next, click the light bulb in the left margin to import the lotus.Domino.JAppletBase package.

Step 5. Update init() method. Change the applet's init() method to notesAppletInit(). Since we've changed our default class from javax.swing.JApplet to lotus.domino.JAppletBase, we need to use this new method to initialize the applet.

Step 6. Add components to the applet. Switch to Design mode (click 'Design' at the top of the source Editor) and drag and drop a JPanel to the applet background; then size it till it covers the entire visual space of applet. Next, add a Start button (JButton) and a progress bar (JProgressBar) to the JPanel. Right-click on the button, click Edit Text and change to name to Start.

That completes getting the basics of the applet itself set up. We'll cover making it functional in Part 3. But next, in Part 2, we're going to import the applet into Notes using Domino Designer and show how to embed it in a Notes Page design item.